An enhanced learning method and system of an accounting intelligent model

Abstract

The application discloses an accounting intelligent model reinforcement learning method and system, and belongs to the field of enterprise financial tax digitization. The system comprises a data acquisition and cleaning layer, an accounting state perception layer, a reinforcement learning decision layer, a business execution layer, a reward and punishment iteration layer and a knowledge sedimentation layer. A deterministic MDP mathematical model coupled with double-entry accounting rules and an accounting tensor state space are constructed in the perception layer. The decision layer adopts an optimized accounting DDQN. A compliance threshold nonlinear reward function is designed in the iteration layer, and a man-machine collaborative incremental learning mechanism is matched. The method comprises seven closed-loop processes of environment initialization, model pre-training, real-time decision, business execution, reward and punishment calculation, autonomous iteration and knowledge sedimentation. The application can independently adapt to new businesses and new financial tax policies, can strengthen the identification ability of high-risk scenes such as fraud and tax violations, and can realize adaptive optimization of the whole process of accounting, risk control, reconciliation and financial decision-making, thereby reducing the cost of financial manual operation and maintenance and the risk of audit compliance.

Description

Technical Field

[0001] This invention relates to the field of enterprise financial and tax digitalization, and in particular to a reinforcement learning method and system for an accounting intelligent model. Background Technology

[0002] Current mainstream intelligent accounting systems are all built using a static rule engine + supervised learning architecture. All business logic relies on manual pre-configuration of accounting standards, business entry templates, and fixed risk control thresholds. Existing technology has four major bottlenecks: Fixed rules with poor adaptability: Manually preset rules cannot automatically adapt to revisions in accounting standards, iterations in business models, and dynamic updates in financial and tax policies. Every policy / business change requires technical personnel to manually modify the rules, resulting in long iteration cycles and high costs. Lacking independent optimization capabilities and low accuracy in handling complex scenarios: The system can only passively execute preset instructions. It has a low fault tolerance rate when facing scenarios such as cross-departmental allocation, multiple borrowing and lending, non-standard and special transactions, discrepancies between accounts and actual assets, and hidden financial and tax risks, and is prone to errors in accounting and omissions in judgment. Multi-objective optimization has compliance compensation defects: Traditional models use linear weighted reward logic, which allows accounting efficiency and processing accuracy to make up for compliance flaws. This conflicts with the rigid requirements of the accounting industry of "compliance veto and zero tolerance for violations", and poses potential risks to tax audit and audit compliance. Insufficient training on high-risk samples: Risk samples such as accounting fraud and large-scale tax violations are sparse, and general reinforcement learning algorithms cannot specifically learn the boundaries of high-risk scenarios, resulting in prominent problems of underreporting and false reporting of risks.

[0003] Existing general reinforcement learning frameworks are not coupled with the underlying rigid constraints of double-entry bookkeeping in accounting, namely "every debit must have a corresponding credit, and debits must equal credits." Their state space and action space are not restricted by financial rules, making it easy for the models to generate illegal decisions such as debit-credit imbalances and irregular journal entries, and thus they cannot be directly applied to accounting professional scenarios.

[0004] In summary, there is currently a lack of a dedicated reinforcement learning technology solution that is deeply aligned with accounting operations, adheres to compliance as a bottom line, and can be independently closed-loop iterated. Furthermore, most general MDP models involve random state transitions, resulting in large training variance and poor stability of accounting decisions, which cannot meet the high-precision and zero-fault-tolerance business requirements of accounting. Summary of the Invention

[0005] Purpose of the invention: The invention provides a reinforcement learning method and system for accounting intelligent models, which solves the defects of existing accounting intelligent systems such as fixed rules, lack of autonomous iteration capability, lack of compliance constraints, multi-objective compensation, and insufficient identification of high-risk scenarios.

[0006] Technical solution: To solve the above-mentioned technical problems, according to one aspect of the present invention, more specifically, a reinforcement learning system for an accounting intelligent model, comprising six hierarchical modules arranged from top to bottom, logically related from beginning to end, and forming a closed loop of information flow: data acquisition and cleaning layer, accounting status intelligent perception layer, reinforcement learning core decision layer, accounting business execution layer, dynamic reward and punishment and iterative update layer, and financial knowledge accumulation and output layer. The data acquisition and cleaning layer is used to uniformly access all the original financial and business data of the enterprise, and output a standardized input dataset after data desensitization, missing value repair, time sequence alignment and business label cleaning. The accounting state intelligent perception layer is configured with a deterministic Markov decision process (MDP) modeling unit under double-entry bookkeeping constraints and an accounting multidimensional state space quantification unit. The deterministic MDP modeling unit constructs a quintuple standardized mathematical model, where each quintuple contains a state space rigidly constrained by the double-entry bookkeeping identity. Hierarchical structured action space Deterministic state transition probability Compliance threshold-based nonlinear reward function Fixed discount factor The aforementioned accounting multidimensional state space quantization unit constructs an accounting state tensor with a debit-credit imbalance penalty index. It quantifies the state of the financial environment from three dimensions: accounting, compliance, and business, and uses compliance constraints to mask irregular accounting conditions with imbalanced lending. The core decision layer of the reinforcement learning is deployed with an improved accounting dual-deep Q network (DDQN). The improved accounting DDQN includes an accounting strategy network and an accounting value network, and has a built-in priority experience playback module and accounting constraint regularization module. The accounting business execution layer connects to the enterprise's ERP system, tax system, and audit system to perform intelligent bookkeeping, automatic reconciliation, risk interception, and financial decision-making actions, and fully records the results of the actions, accounting deviations, and risk feedback data. The dynamic reward and punishment and iterative update layer is configured with a compliance threshold-based nonlinear reward calculation unit, which calculates reward and punishment values ​​based on business execution feedback and drives the improved accounting DDQN to complete the iterative update of network parameters. The financial knowledge accumulation and output layer is used to solidify the optimal accounting treatment strategy and risk control rules obtained after model iteration and convergence, forming a unique financial knowledge graph for the enterprise. It also supports lightweight model export, business scenario replication and cross-enterprise reuse. The solidified knowledge in turn assists the core decision-making layer in optimizing accounting strategies. The six-layer module forms a complete autonomous evolution closed-loop architecture through data interaction, parameter iteration, and knowledge reuse.

[0007] Furthermore, the accounting state tensor constructed by the accounting multidimensional state space quantization unit... The mathematical expression is:

[0008] in, This is a dynamic account balance matrix at time t, storing the debit and credit entries and ending balances of all accounting accounts. Let t be the original feature vector of the business to be processed at time t; The penalty amplification factor for lending imbalance; The L2 norm squared is used to quantify the degree of deviation from the double-entry bookkeeping identity, representing the degree of imbalance in debit and credit entries.

[0009] Furthermore, the total return calculation formula for the compliance threshold-based nonlinear reward function is as follows:

[0010] in, A binary indicator function, representing a hard threshold for compliance and security. This serves as a veto criterion for accounting compliance; These are the positive weighting coefficients for accounting accuracy and risk avoidance, respectively. This is a violation depth indicator, representing the severity of accounting and tax violations; It experienced exponential explosive growth; The independent penalty amplification factor, and the discount factor Independent of each other; when the accounting compliance score is below the threshold When the indicator function is set to 0, all positive rewards are cleared, the system enters penalty exclusive mode, and only outputs the exponential penalty value that is uniquely determined by the depth of the violation.

[0011] The weighted deduction scoring system based on multi-dimensional rule layers is calculated using the following formula: : A pre-defined set of mandatory compliance rules (subject account usage guidelines, invoice verification rules, tax law provisions, internal risk control systems, etc.); : No. The penalty weight for violating the rules (if a rigid rule is violated, then...) Extremely high; if the suggested rule is violated, then... (Smaller). Designed for >1. Ensure that any rigid violation will cause the score to drop directly below the 0.99 threshold.

[0012] Furthermore, the hierarchical structured action space It is divided into top-level business semantic intent actions and bottom-level accounting correction actions; the top-level business semantic intent actions include depreciation accrual, revenue recognition, expense amortization, accounts receivable / payable reconciliation, and standardized accounting adjustment operations; the bottom-level accounting correction actions are the increments of debit and credit amounts for each accounting item. And set rigid constraints This ensures that every output action naturally satisfies the borrowing and lending balance rule.

[0013] Furthermore, the dynamic reward and punishment and iterative update layer is also equipped with a human-machine collaborative incremental learning unit; the human-machine collaborative incremental learning unit is used to receive manual correction samples from financial experts for new business, new financial and tax policies, and difficult accounting issues, and to transform the experience of human experts into incremental samples to input into the improved accounting DDQN, so that the model can be quickly adapted without full retraining.

[0014] Furthermore, the state transition probabilities in the deterministic MDP modeling unit For deterministic transition mechanisms; given the current state. With output action The only way to determine the accounting status at the next moment. This eliminates random environmental interference, reduces the variance of model Q-value estimation, and improves the stability and accuracy of accounting decisions.

[0015] A reinforcement learning method for an accounting intelligence model, applied to the reinforcement learning system of the accounting intelligence model, the method comprising the following closed-loop iterative steps: S1. Environment Initialization: After accessing the enterprise's full financial and business data through the data acquisition and cleaning layer and completing the standardized cleaning process, the initial accounting state space and hierarchical structured action space are constructed. S2, Model Pre-training: Based on the enterprise's historical accounting, risk control, and audit samples, complete the initial training of the improved accounting DDQN to generate basic accounting business processing strategies; S3. Real-time Status Awareness and Intelligent Decision-Making: The accounting status intelligent awareness layer quantifies the current financial environment in real time to obtain the accounting status tensor. After filtering out illegal states of loan imbalance, the input is given to the improved accounting DDQN in the core decision layer of reinforcement learning, and the output is the compliant optimal accounting operation action; S4. Business Implementation: The accounting business execution layer executes the output of bookkeeping, risk control, reconciliation, and financial decision-making actions, and fully records the execution results, accounting deviations, and risk feedback data. S5. Dynamic reward and punishment calculation: The dynamic reward and punishment and iterative update layer calls the compliance threshold-type non-linear reward function, with 0.99 as the hard compliance judgment threshold. If the compliance score meets the standard, the positive benefits of accounting accuracy, cost control and risk prevention are calculated comprehensively. If the compliance score does not meet the standard, it enters the penalty exclusive mode, and the exponential penalty value is calculated only based on the depth of the violation. S6. Model Autonomous Iterative Update: The strategy network and value network parameters of the improved accounting DDQN are updated based on reward and punishment feedback gradient; for new business and new financial and tax policy scenarios, samples manually corrected by financial experts are imported to perform incremental learning. S7. Financial Knowledge Consolidation and Deposition: The financial knowledge deposition and output layer consolidates and updates the optimal accounting treatment strategies and risk control rules after iterative convergence into the financial knowledge graph; steps S3 to S7 are executed cyclically to achieve continuous autonomous optimization and iteration of the system.

[0016] Furthermore, in step S3, when quantifying the financial status, a penalty indicator for debit and credit imbalance is generated based on the double-entry bookkeeping identity. The illegal accounting status of debit and credit imbalance is directly filtered out through a compliance constraint mask, prohibiting the model from generating accounting decision actions based on illegal status.

[0017] Furthermore, in the reward and punishment calculation process of step S5, a compliance score threshold of 0.99 is used as a prerequisite hard constraint for entry, and a compliance veto mechanism is strictly implemented. When compliance is not met, all positive benefits are cleared. The severity of the punishment is determined solely by the severity of the violation and is unrelated to the specific value of the compliance score.

[0018] Furthermore, during the model iteration process in step S6, the training weights of scarce and high-risk samples such as accounting errors, tax risks, and reconciliation anomalies are increased through a priority experience replay mechanism; at the same time, the accounting constraint regularization module is used to suppress model strategy drift and overfitting, ensuring accounting compliance throughout the iteration process.

[0019] Beneficial effects: (1) This invention constructs a closed-loop architecture of full-link autonomous evolution consisting of a data collection and cleaning layer, an intelligent perception layer of accounting status, a core decision-making layer of reinforcement learning, an accounting business execution layer, a dynamic reward and punishment and iterative update layer, and a financial knowledge accumulation and output layer. It is equipped with a human-machine collaborative incremental learning unit, which can directly access the manual correction samples issued by financial experts for new financial and tax policies, new businesses, and difficult accounting issues to complete incremental training. There is no need to carry out full retraining of the model, and no need for technical personnel to continuously manually modify and maintain the journal entry templates and risk control rules. It can quickly adapt to scenarios of accounting standard revision, enterprise business iteration, and dynamic adjustment of financial and tax policies, greatly shortening the system iteration cycle and reducing the manual operation and maintenance costs of financial digitalization. (2) This invention constructs an accounting multidimensional state tensor containing the L2 norm penalty index for debit and credit imbalance in the intelligent perception layer of accounting status. It directly filters illegal accounting states with debit and credit imbalance by means of compliance constraint mask. At the same time, it designs a hierarchical structured action space and adds a rigid balance constraint for the incremental debit and credit amount. It adopts a deterministic MDP state transition mechanism. The current financial status matches the corresponding accounting action to uniquely determine the accounting status at the next moment. It eliminates the model Q value estimation variance caused by random environment. It natively binds the underlying rigid rule of double-entry bookkeeping "for every debit there must be a credit, and debits must be equal" from the entire link of state quantification, action output and state transition. It completely overcomes the industry defects of general reinforcement learning models that lack financial professional constraints and are prone to generating illegal entries with debit and credit imbalance. It effectively improves the stability of accounting decisions and the accuracy of accounting processing. (3) This invention has created a unique compliance threshold-type nonlinear reward function, which sets a hard compliance score threshold of 0.99 to achieve compliance veto control. If the accounting compliance score does not reach the threshold, all positive income will be directly cleared. Only the violation depth index is used to apply exponential penalties, which completely eliminates the compliance compensation defects of the traditional linear weighted reward mechanism. It fully matches the rigid regulatory requirements of zero tolerance for violations in the accounting industry's tax inspection and audit verification. The exponential gradient penalty can accurately distinguish the severity level of accounting and tax violations, and reduce the compliance risks of corporate tax penalties and audit rectification from the algorithm bottom layer. (4) The present invention incorporates a priority experience playback module and an accounting constraint regularization module in the improved accounting DDQN. The former increases the training weight of scarce high-risk samples such as accounting fraud, large-scale tax violations, discrepancies between accounts and actual situation, and abnormal reconciliation, thereby strengthening the model's ability to identify hidden high-risk scenarios and reducing the problems of risk underreporting and false reporting. The latter constrains the accounting compliance logic during the entire iteration process of the model, suppresses model strategy drift and overfitting, and has a higher processing fault tolerance rate for complex financial scenarios that are difficult for traditional systems to adapt to, such as multiple debits and credits, cross-departmental expense allocation, and non-standard special transactions. (5) This invention solidifies the optimal accounting treatment strategy and risk control rules formed after model iteration and convergence through the financial knowledge accumulation and output layer, generating an enterprise-specific financial knowledge graph. The accumulated professional knowledge can flow back to the core decision-making layer of reinforcement learning to continuously optimize the accounting treatment strategy. At the same time, it supports lightweight model export, business scenario replication and cross-enterprise reuse, transforming the financial intelligence capabilities formed by iterative optimization into reusable enterprise digital assets, significantly reducing the secondary development and deployment costs of various enterprise financial and tax intelligence systems. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the system framework of the present invention; Figure 2 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0021] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Reference Figure 1 This is a reinforcement learning system for an accounting intelligent model. From top to bottom, it consists of six layers: a data acquisition and cleaning layer, an intelligent accounting status perception layer, a reinforcement learning core decision-making layer, an accounting business execution layer, a dynamic reward and punishment and iterative update layer, and a financial knowledge accumulation and output layer. The six-layer architecture forms a logically connected information flow loop, realizing full-link autonomous optimization of data iteration, parameter updates, and knowledge accumulation. (1) Data Acquisition and Cleaning Layer It unifies the access to multi-source data from ERP accounting, bank statements, invoices and taxes, expense reimbursements, business contracts, audit history, and financial and tax policy texts; performs data anonymization, missing value imputation, time-series alignment, and business labeling processing, removes noisy data, and outputs a standardized structured dataset as input for the reinforcement learning environment.

[0023] (2) Accounting Status Intelligent Perception Layer It comprises two core units: a multidimensional state-space quantization unit and a double-entry bookkeeping constraint deterministic MDP modeling unit.

[0024] ① Standardized modeling of accounting state tensors: Define the global state tensor at time t. ; : Account balance matrix, storing all debit and credit transactions and ending balances for all accounts; Business feature vectors aggregate business information such as transaction type, counterparty, and fee attributes; : A penalty indicator for borrowing and lending imbalance, which quantifies the deviation from the double-entry bookkeeping identity, where λ is a configurable amplification factor; At the same time, a compliance constraint mask is added to directly block illegal situations of lending imbalance.

[0025] ② Customized deterministic MDP quintuple constraint modeling: state space : Contains only compliant accounting tensors that balance borrowing and lending; Action space The structure is a two-tiered system. The top tier represents business operations (accrual, revenue recognition, reconciliation adjustments, etc.), while the bottom tier represents the incremental debit and credit entries (Δ) of accounts, with mandatory constraints. ; State transition Deterministic transition: The same state + action uniquely determines the next accounting state, eliminating random environmental interference, reducing the variance of model Q-value estimation, and improving training stability; reward function Compliance threshold non-linear reward and punishment mechanism; Discount factor The fixed factor is the discount factor for the long-term returns of reinforcement learning, balancing long-term audit compliance benefits with short-term bookkeeping efficiency.

[0026] (3) Enhance the core decision-making level of learning Deploy an improved accounting dual-depth Q network (DDQN), including an accounting strategy network and an accounting value network; built-in: 1) Priority Experience Replay Module: Increases the sampling weight of scarce and high-risk samples such as accounting errors, tax anomalies, and financial fraud; 2) Accounting Constraint Regularization Module: Suppresses model overfitting and policy drift, ensuring accounting compliance throughout the iteration process.

[0027] The strategy network outputs the optimal accounting action, while the value network evaluates the long-term comprehensive return of the action and corrects strategy bias.

[0028] (4) Accounting business execution level It connects to the enterprise's financial, tax, and audit systems to perform operations such as automatic voucher generation, intelligent account adjustment, risk interception, and funding plan recommendation; it fully records the execution results of each action, accounting deviations, and risk feedback, providing raw data for reward and punishment calculations.

[0029] (5) Dynamic reward and punishment and iterative update layer Configure a compliance threshold-based non-linear reward calculation unit; reward formula:

[0030] Parameter description: Binary indicator function, hard threshold for compliance and security This serves as the veto standard for determining compliance with accounting regulations. For accuracy and risk-return weighting factors; This is a violation depth indicator, representing the severity of accounting and tax violations; For independent penalty amplification coefficients, and discount factors Completely distinguish the physical meanings and avoid symbolic conflicts; When the compliance score falls below the threshold, the system enters a penalty-exclusive mode, all positive benefits are reset to zero, and the severity of the penalty is determined solely by the depth of the violation. The decision is made independently and is unrelated to the specific value of the compliance score. The hard constraint of "one vote for veto" in accounting compliance is strictly implemented, resulting in minor penalties for minor violations and exponentially high penalties for major violations. It also has a built-in human-machine collaborative incremental learning unit that receives manual correction samples from financial experts for new business and difficult accounting entries, and can complete incremental updates without full retraining.

[0031] (6) Financial knowledge accumulation and output layer The optimal journal entry strategy and risk identification rules after model iteration convergence are solidified into a corporate-specific financial knowledge graph; the model can be exported in a lightweight manner and replicated in a scenario, enabling the intelligent capabilities to be reused across enterprises, and the solidified knowledge can be used to assist the decision-making level in optimizing subsequent accounting decisions.

[0032] Reference Figure 2 A reinforcement learning system for an accounting intelligence model includes the following specific implementation process: Step S1 Environment Initialization: Access all financial business data, and construct the initial accounting state and action space after standardization and cleaning; Step S2 Model Pre-training: Based on historical accounting, auditing, and risk samples, complete the improved DDQN pre-training to generate basic accounting processing strategies; Step S3 Real-time perception and intelligent decision-making: The perception layer calculates the accounting state tensor in real time, filters illegal and unbalanced states, inputs DDQN, and outputs compliant accounting actions; Step S4 Business Implementation: Perform accounting, risk control, and reconciliation operations, and record all execution feedback data; Step S5 Dynamic reward and punishment calculation: Call the non-linear reward function to determine the compliance level. If the compliance is met, the positive benefits are calculated comprehensively. If the compliance is not met, the penalty exclusive mode is entered, and the exponential penalty value is calculated only based on the depth of the violation. Step S6: Model Autonomous Iteration: Update DDQN network parameters based on reward and punishment gradients; Incremental learning is performed by importing expert-corrected samples for new policies and new business scenarios; Step S7 Knowledge Consolidation and Accumulation: Update the optimal strategy to the financial knowledge graph; repeat steps S3-S7 to achieve continuous autonomous system improvement. The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A reinforcement learning system for an accounting intelligent model, comprising a data acquisition and cleaning layer, an accounting status intelligent perception layer, a reinforcement learning core decision-making layer, an accounting business execution layer, a dynamic reward and punishment and iterative update layer, and a financial knowledge accumulation and output layer, characterized in that: The data acquisition and cleaning layer is used to uniformly access all the original financial and business data of the enterprise, and output a standardized input dataset after data desensitization, missing value repair, time sequence alignment and business label cleaning. The accounting state intelligent perception layer is configured with a deterministic Markov decision process (MDP) modeling unit under double-entry bookkeeping constraints and an accounting multidimensional state space quantification unit. The deterministic MDP modeling unit constructs a quintuple standardized mathematical model, where each quintuple contains a state space rigidly constrained by the double-entry bookkeeping identity. Hierarchical structured action space Deterministic state transition probability Compliance threshold-based nonlinear reward function Fixed discount factor The aforementioned accounting multidimensional state space quantization unit constructs an accounting state tensor with a debit-credit imbalance penalty index. It quantifies the state of the financial environment from three dimensions: accounting, compliance, and business, and uses compliance constraints to mask irregular accounting conditions with imbalanced lending. The core decision layer of the reinforcement learning is deployed with an improved accounting dual-deep Q network (DDQN). The improved accounting DDQN includes an accounting strategy network and an accounting value network, and has a built-in priority experience playback module and accounting constraint regularization module. The accounting business execution layer connects to the enterprise's ERP system, tax system, and audit system to perform intelligent bookkeeping, automatic reconciliation, risk interception, and financial decision-making actions, and fully records the results of the actions, accounting deviations, and risk feedback data. The dynamic reward and punishment and iterative update layer is configured with a compliance threshold-based nonlinear reward calculation unit, which calculates reward and punishment values ​​based on business execution feedback and drives the improved accounting DDQN to complete the iterative update of network parameters. The financial knowledge accumulation and output layer is used to solidify the optimal accounting treatment strategy and risk control rules obtained after model iteration and convergence, forming a unique financial knowledge graph for the enterprise. It also supports lightweight model export, business scenario replication and cross-enterprise reuse. The solidified knowledge in turn assists the core decision-making layer in optimizing accounting strategies. The six-layer module forms a complete autonomous evolution closed-loop architecture through data interaction, parameter iteration, and knowledge reuse.

2. The reinforcement learning system for an accounting intelligent model according to claim 1, characterized in that, The accounting state tensor constructed by the accounting multidimensional state space quantization unit The mathematical expression is: in, This is a dynamic account balance matrix at time t, storing the debit and credit entries and ending balances of all accounting accounts. Let t be the original feature vector of the business to be processed at time t; The penalty amplification factor for lending imbalance; The L2 norm squared is used to quantify the degree of deviation from the double-entry bookkeeping identity, representing the degree of imbalance in debit and credit entries.

3. The reinforcement learning system for an accounting intelligent model according to claim 1, characterized in that, The formula for calculating the total return of the compliance threshold-based nonlinear reward function is as follows: in, A binary indicator function, representing a hard threshold for compliance and security. This serves as a veto criterion for accounting compliance; These are the positive weighting coefficients for accounting accuracy and risk avoidance, respectively. This is a violation depth indicator, representing the severity of accounting and tax violations; It experienced exponential explosive growth; The independent penalty amplification factor, and the discount factor Independent of each other; when the accounting compliance score is below the threshold When the indicator function is set to 0, all positive earnings are cleared, the system enters penalty exclusive mode, and only outputs the exponential penalty value that is uniquely determined by the depth of the violation. The weighted deduction scoring system based on multi-dimensional rule layers is calculated using the following formula: A pre-defined set of mandatory compliance rules; : No. The point deduction weight for violating the rules.

4. The reinforcement learning system for an accounting intelligent model according to claim 1, characterized in that, The hierarchical structured action space It is divided into top-level business semantic intent actions and bottom-level accounting correction actions; the top-level business semantic intent actions include depreciation accrual, revenue recognition, expense amortization, accounts receivable / payable reconciliation, and standardized accounting adjustment operations; the bottom-level accounting correction actions are the increments of debit and credit amounts for each accounting item. And set rigid constraints This ensures that every output action naturally satisfies the borrowing and lending balance rule.

5. The reinforcement learning system for an accounting intelligent model according to claim 1, characterized in that, The dynamic reward and punishment and iterative update layer is also configured with a human-machine collaborative incremental learning unit; the human-machine collaborative incremental learning unit is used to receive manual correction samples from financial experts for new business, new financial and tax policies, and difficult accounting issues, and to convert the experience of human experts into incremental samples to input into the improved accounting DDQN, so that the model can be quickly adapted without full retraining.

6. The reinforcement learning system for an accounting intelligent model according to claim 1, characterized in that, The state transition probability in the deterministic MDP modeling unit For deterministic transition mechanisms; given the current state. With output action The only way to determine the accounting status at the next moment. This eliminates random environmental interference, reduces the variance of model Q-value estimation, and improves the stability and accuracy of accounting decisions.

7. A reinforcement learning method for an accounting intelligent model, characterized in that, A reinforcement learning system applied to the accounting intelligent model according to any one of claims 1-7, the method comprising the following closed-loop iterative steps: S1. Environment Initialization: After accessing the enterprise's full financial and business data through the data acquisition and cleaning layer and completing the standardized cleaning process, the initial accounting state space and hierarchical structured action space are constructed. S2. Model pre-training: Based on the company's historical accounting, risk control, and audit samples, complete the initial training of the improved accounting DDQN to generate basic accounting business processing strategies; S3. Real-time Status Awareness and Intelligent Decision-Making: The accounting status intelligent awareness layer quantifies the current financial environment in real time to obtain the accounting status tensor. After filtering out illegal states of loan imbalance, the input is given to the improved accounting DDQN in the core decision layer of reinforcement learning, and the output is the compliant optimal accounting operation action; S4. Business Implementation: The accounting business execution layer executes the output of bookkeeping, risk control, reconciliation, and financial decision-making actions, and fully records the execution results, accounting deviations, and risk feedback data. S5. Dynamic reward and punishment calculation: The dynamic reward and punishment and iterative update layer calls the compliance threshold-type non-linear reward function, with 0.99 as the hard compliance judgment threshold. If the compliance score meets the standard, the positive benefits of accounting accuracy, cost control and risk prevention are calculated comprehensively. If the compliance score does not meet the standard, it enters the penalty exclusive mode, and the exponential penalty value is calculated only based on the depth of the violation. S6. Model Autonomous Iterative Update: The strategy network and value network parameters of the improved accounting DDQN are updated based on the reward and punishment feedback gradient; for new business and new financial and tax policy scenarios, samples manually corrected by financial experts are imported to perform incremental learning; S7. Financial Knowledge Consolidation and Deposition: The financial knowledge deposition and output layer consolidates and updates the optimal accounting treatment strategies and risk control rules after iterative convergence into the financial knowledge graph; steps S3 to S7 are executed cyclically to achieve continuous autonomous optimization and iteration of the system.

8. The reinforcement learning method for an accounting intelligent model according to claim 7, characterized in that, In step S3, when quantifying the financial status, a penalty indicator for debit and credit imbalance is generated based on the double-entry bookkeeping identity. The illegal accounting status of debit and credit imbalance is directly filtered out through a compliance constraint mask, prohibiting the model from generating accounting decision actions based on illegal status.

9. The reinforcement learning method for an accounting intelligent model according to claim 7, characterized in that, In the reward and punishment calculation process of step S5, a compliance score threshold of 0.99 is used as a prerequisite hard constraint for admission. The compliance veto mechanism is strictly implemented. When compliance is not met, all positive benefits are cleared. The severity of the punishment is determined independently by the severity of the violation and is not related to the specific value of the compliance score.

10. The reinforcement learning method for an accounting intelligent model according to claim 7, characterized in that, During the model iteration process in step S6, the training weights of scarce and high-risk samples such as accounting errors, tax risks, and reconciliation anomalies are increased through the priority experience replay mechanism; at the same time, the accounting constraint regularization module is used to suppress model strategy drift and overfitting, ensuring accounting compliance throughout the iteration process.

Images