A large model optimization method for finance and tax numerical calculation

Abstract

The present application relates to the technical field of intelligent financial and tax calculation, in particular to a large model optimization method for financial and tax numerical calculation, comprising the following steps: constructing a fine-tuning data set of financial and tax calculation instructions containing numerical sources, calculation logic and policy basis; fusing numerical values and units into joint coding by numerical unit binding embedded units and adding precision identification bits; using an improved encoder introducing numerical semantic attention weight adjustment factors; and supervising and fine-tuning the pre-constructed large model based on the fine-tuning data set. In the present application, the magnitude confusion and precision loss problem is solved by deeply binding numerical values and units, the attention of numerical features and policy keywords is dynamically adjusted by using learnable parameters, the sensitivity of the model to financial and tax rules is strengthened, and the accuracy, compliance and logical rigor of numerical calculation of the model in complex financial and tax scenarios are significantly improved by training combined with a full-link labeled professional data set.

Description

Technical Field

[0001] This invention relates to the field of intelligent financial and tax computing technology, and in particular to a large-scale model optimization method for numerical financial and tax computing. Background Technology

[0002] Intelligent financial and tax computing refers to the technical field of using artificial intelligence technology to assist in handling complex numerical calculations in scenarios such as tax declaration, financial accounting, and report analysis. This field involves high-precision tasks such as value-added tax deduction and income tax settlement, requiring the system to not only possess natural language understanding capabilities but also strictly adhere to dynamically changing tax policies and regulations. Traditional large-scale model technology refers to large-scale language models based on the Transformer architecture, primarily relying on self-supervised training with massive amounts of general-purpose text. It generates text by capturing statistical patterns between word units, and its core advantage lies in general semantic understanding and content generation.

[0003] However, when existing general-purpose models are directly applied to fiscal and tax numerical calculations, their word segmentation mechanisms often separate numerical values ​​from units, making it difficult for the model to perceive the difference in magnitude between "ten thousand yuan" and "yuan." This can easily lead to serious numerical confusion and loss of precision. At the same time, the existing architecture lacks explicit constraints on specific fiscal and tax calculation logic and policy rules, making it easy for the model to ignore the restrictions of tax law provisions (such as pre-tax deduction limits) when generating calculation steps, resulting in logical jumps or compliance errors. In addition, the general training data lacks high-quality fiscal and tax labeled samples containing complete calculation reasoning chains, making it impossible for the model to learn accurate numerical reasoning paradigms and failing to meet the stringent requirements of zero error tolerance in the fiscal and tax field. Summary of the Invention

[0004] To address the technical problems existing in the prior art, this invention provides a large-scale model optimization method for numerical calculations in finance and taxation. The technical solution is as follows: On the one hand, a large-scale model optimization method for fiscal and tax numerical calculations is provided, including the following steps: By analyzing historical cases of corporate tax declarations and financial audits, five-dimensional features—input text, numerical values, calculation steps, correct results, and regulatory basis—are extracted to construct a fine-tuning dataset of financial and tax calculation instructions. Obtain the sequence of financial and tax texts to be processed, identify the numerical values ​​and units of measurement in the sequence of financial and tax texts through the numerical unit binding embedding unit, merge the numerical values ​​and the adjacent units of measurement into a numerical unit joint word, add a precision flag bit to the numerical unit joint word, and generate a fusion feature embedding vector. The fused features are embedded into a vector and input into an improved Transformer encoder. In the self-attention calculation layer, a numerical semantic attention weight adjustment factor is introduced to generate a corrected attention score based on a dynamic weight calculation formula, and numerical semantic association features in the financial and tax text sequence are extracted accordingly. Based on the aforementioned tax calculation instructions, the dataset is fine-tuned, and the difference between the generated results and the standard answer is calculated using the cross-entropy loss function. The model parameters are then updated through backpropagation to complete the supervised adjustment of the basic large model.

[0005] On the other hand, the process of generating the fused feature embedding vector specifically includes: Establish a dedicated unit dictionary containing commonly used financial and tax measurement units, traverse the financial and tax text sequence, and when a numerical field is detected and is immediately followed by a unit field in the dedicated unit dictionary, map the numerical field and the unit field to a unique numerical unit joint term. Calculate the number of significant digits after the decimal point in the numerical field, map the significant digits to the corresponding positional encoding vector as the precision identifier, and superimpose the precision identifier into the word embedding vector of the numerical unit joint lexical.

[0006] On the other hand, the specific process of generating the corrected attention score based on the dynamic weight calculation formula adopts the following formula: ; in, This represents the corrected attention score. This represents the original base attention score calculated based on the dot product of the query vector and the key vector. This represents a learnable, adjustable parameter for numerical features. The feature weight values ​​represent the numerical unit joint words in the input sequence. This represents a learnable, adjustable parameter based on the characteristics of the specification. The feature weight values ​​represent the keywords related to financial and tax regulations in the input sequence.

[0007] On the other hand, the learnable adjustable parameters are given preset non-zero initial values ​​during the model initialization phase, and are dynamically updated during the supervised adjustment process as the gradient descent direction of the cross-entropy loss function changes.

[0008] On the other hand, the process of constructing the fine-tuning dataset for financial and tax calculation instructions specifically includes: Unstructured text is extracted from the original financial and tax documents, and the amount, tax rate, and time values ​​are extracted as a set of key values ​​using preset regular expressions. Based on the provisions of current tax laws, the calculation process from the set of key values ​​to the final result is parsed into a logical chain described in natural language, which serves as the calculation steps; The corresponding tax law clause chapter code is used as the standard basis, and the input text, the numerical value, the calculation steps, the correct result and the standard basis are formatted into training samples according to the preset instruction template.

[0009] On the other hand, the calculation logic of the cross-entropy loss function is specifically limited as follows: Only the differences between the sequence of calculation steps and the final result value generated by the model and the standard annotation in the fine-tuning dataset of the financial and tax calculation instructions are calculated, ignoring the prediction loss of the input text portion.

[0010] On the other hand, when the improved Transformer encoder processes ordinary text lexical units that are not numerical or non-canonical keywords, it automatically sets the parameters corresponding to the ordinary text lexical units to zero, so that the corrected attention score reverts to the original basic attention score.

[0011] On the other hand, the method also includes a simulated sample augmentation step: Based on known financial and tax calculation formula templates, by randomly generating values ​​that conform to the business logic range and replacing the variables in the financial and tax calculation formula templates, a batch of simulated financial and tax calculation cases are automatically generated, and the simulated financial and tax calculation cases are mixed into the financial and tax calculation instruction fine-tuning dataset.

[0012] On the other hand, the sequence of financial and tax documents includes notes to the enterprise value-added tax return, explanations to the enterprise income tax settlement report, and explanations to the financial statements.

[0013] On the other hand, the output of the method includes numerical calculation results and corresponding calculation logic explanation text, in which the names of tax law clauses used as the basis for the calculation are explicitly referenced.

[0014] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: By introducing a numerical unit binding mechanism and dynamic attention weight adjustment, the underlying reconstruction of numerical computing capabilities in the field of finance and taxation has been achieved. The design of numerical unit joint terms completely eliminates the model's cognitive bias of numerical magnitude. Combined with precision identifiers, it ensures the decimal accuracy of the calculation results. By embedding learnable adjustment parameters into the attention mechanism, the model can adaptively focus on key numerical values ​​and policy and regulatory provisions, effectively suppressing the interference of irrelevant semantics and ensuring that the calculation logic strictly follows tax law regulations. Relying on the supervision and fine-tuning of full-link labeled data, the model is endowed with a complete closed loop of capability from policy and regulatory understanding to numerical reasoning, which significantly reduces compliance risks and manual review costs in tax declaration and financial accounting. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of the main steps of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0019] This invention provides a method for optimizing large-scale models for numerical calculations in finance and taxation, such as... Figure 1 As shown, it includes the following steps: S1: By analyzing historical cases of corporate tax declarations and financial audits, five-dimensional features are extracted, including input text, numerical values, calculation steps, correct results, and regulatory basis, to construct a fine-tuning dataset for financial and tax calculation instructions.

[0020] The method also includes a simulated sample enhancement step: based on the known financial and tax calculation formula template, by randomly generating values ​​that conform to the business logic range and replacing the variables in the financial and tax calculation formula template, a batch of simulated financial and tax calculation cases are automatically generated, and the simulated financial and tax calculation cases are mixed into the financial and tax calculation instruction fine-tuning dataset.

[0021] The process of constructing a fine-tuning dataset for tax and financial calculation instructions specifically includes: extracting unstructured text from the original tax and financial documents, using preset regular expressions to extract amounts, tax rates, and time values ​​as a set of key values; based on current tax law provisions, parsing the calculation process from the set of key values ​​to the final result into a logical chain described in natural language, as the calculation steps; encoding the corresponding tax law provisions into chapters as the standard basis, and formatting the input text, values, calculation steps, correct results, and standard basis into training samples according to a preset instruction template.

[0022] In the process of constructing the fine-tuning dataset for financial and tax calculation instructions, a high-precision data processing pipeline is first established. This pipeline directly connects to the enterprise's ERP system exported files, the tax bureau's historical backup data of declarations, and the electronic archives of audit working papers. To ensure the diversity and coverage of the data, the selected data sources must cover no less than 5,000 annual financial statements and tax returns from different industries (such as manufacturing, service, and high-tech enterprises). In the data preprocessing stage, OCR (Optical Character Recognition) technology is used to digitize scanned documents (such as PDF format VAT invoices and stamped audit reports). The recognition accuracy rate must be set above 99.9%. Character segments with a confidence level below 95% are automatically marked as pending review.

[0023] In the process of extracting the "five-dimensional features," the first step is to segment the lengthy audit notes into independent semantic paragraphs using sentence segmentation algorithms in Natural Language Processing (NLP) for the "input text" features. Next, for the extraction of "numerical" features, the process does not rely solely on simple number lookups but instead constructs a complex regular expression library. For example, for monetary data, pattern matching rules are used to identify number strings containing currency units such as "yuan," "ten thousand yuan," and "CNY," while removing interference from non-monetary numbers such as phone numbers, postal codes, or dates. For tax rates, the focus is on capturing floating-point numbers or integers before the "%" symbol and confirming them in conjunction with contextual keywords (such as "applicable tax rate" and "levy rate").

[0024] For the construction of the "calculation steps" feature, a reverse analysis method is adopted. Assuming that the known input items are "operating revenue" and "operating costs", and the "correct result" is "operating profit", a logical chain described in natural language is automatically generated based on a pre-set accounting standard logic library. For example: "Step 1, read the operating revenue value; Step 2, read the operating cost value; Step 3, perform the subtraction operation, that is, subtract the operating cost from the operating revenue to obtain the operating profit". If complex corporate income tax settlement is involved, the logical chain will be further refined, including intermediate links such as "additional deduction of R&D expenses" and "exclusion of tax-exempt income". For the "standard basis" feature, by establishing a full tax law knowledge graph, the corresponding legal provisions on which each calculation is based are structured and encoded to ensure that each calculation step has a basis.

[0025] In the simulation sample enhancement stage, to address the scarcity of extreme cases (such as huge losses, extremely high profits, and special tax incentive combinations) in financial and tax data, a template-based generation engine was designed. This engine has more than 200 built-in standard financial and tax calculation formula templates. For example, for the calculation formula of "transfer of input VAT", the engine sets a reasonable range of variables: the raw material cost is set between 100,000 yuan and 50 million yuan and conforms to a log-normal distribution; the abnormal loss ratio is set between 1% and 100%. Through the Monte Carlo sampling method, tens of thousands of sets of numerical combinations that conform to business logic are randomly generated and filled into the template to generate new virtual cases. The simulated cases expand the data scale and enhance the robustness of the model by introducing boundary values ​​(such as 0 yuan, negative numbers, and maximum values).

[0026] A typical set of VAT declaration data is selected for processing and illustration, as shown in Table 1 below, which demonstrates an example of extracting key elements from the original unstructured text and transforming them into structured training samples.

[0027] Table 1. Sample Construction Table for Fine-tuning Dataset of Financial and Tax Calculation Instructions 1 This month's sales revenue was 500,000 yuan, and the applicable tax rate is 13%. 500000；13 Step 1: Identify sales amount; Step 2: Identify tax rate; Step 3: Calculate output tax = sales amount × tax rate TS_VAT_001 (Provisional Regulations on Value Added Tax) 2 Freight expenses of 2,000 yuan were incurred, and a special VAT invoice was obtained with a tax amount of 180 yuan. 2000；180 Step 1: Identify the freight amount; Step 2: Identify the input tax amount; Step 3: Confirm the deductible amount as 180. TS_VAT_005 (Input Tax Deduction Regulations) 3 Goods purchased for collective welfare amounted to 3,000 yuan, with an input tax of 390 yuan. 3000；390 Step 1: Identify the purchase amount; Step 2: Determine the purpose as collective welfare; Step 3: Determine if the purchase is not deductible. TS_VAT_008 (Regulations for Transferring Income Tax Out) As shown in Table 1, through the above processing, the originally discrete text information is transformed into machine-understandable structured instructions. Each sample fully contains the reasoning process from input to output, providing a high-quality data foundation for the supervised fine-tuning of subsequent large models.

[0028] S2: Obtain the sequence of financial and tax texts to be processed, identify the numerical values ​​and units of measurement in the financial and tax text sequence through the numerical unit binding embedding unit, merge the numerical values ​​and their adjacent units of measurement into numerical unit joint words, add a precision flag bit to the numerical unit joint words, and generate a fusion feature embedding vector.

[0029] The series of financial and tax documents includes notes to the enterprise value-added tax return, instructions for the enterprise income tax settlement report, and notes to the financial statements.

[0030] The process of generating fusion feature embedding vectors specifically includes: establishing a dedicated unit dictionary containing commonly used financial and tax measurement units; traversing the financial and tax text sequence; when a numerical field is detected and is immediately followed by a unit field in the dedicated unit dictionary, mapping the numerical field and the unit field to a unique numerical unit joint term; calculating the number of significant digits after the decimal point in the numerical field, mapping the significant digits to the corresponding positional encoding vector as a precision identifier, and superimposing the precision identifier into the word embedding vector of the numerical unit joint term.

[0031] After acquiring the sequence of financial and tax texts to be processed, the "numerical unit binding" operation is performed. This aims to solve the common problem of traditional large models breaking down numbers into meaningless sub-words, thus losing numerical precision. Specifically, when reading a text stream such as "Explanation of Corporate Income Tax Settlement Report", the preprocessing module first starts the word segmenter. The word list here has been specifically modified. Instead of splitting "1,234,567.89" into fragments such as "1", "," and "234", it is treated as a whole tag. At the same time, the text window before and after the value (the window size is set to 5 words) is scanned in parallel to find the matching unit of measurement.

[0032] In the process of generating the fusion feature embedding vector, a dedicated unit dictionary is first loaded. This dictionary covers more than 95% of commonly used units in the financial and tax field, including currency units (such as "RMB", "USD", "thousand yuan"), proportion units (such as "%", "‰"), time units (such as "accounting year", "quarter"), and physical measurement units (such as "unit", "set", "ton"). The system traverses the text sequence. Once a numerical field is detected using the regular expression rule \d+(\.\d+)?, it immediately searches the text immediately to its right. If the unit field "ten thousand yuan" is found, the system's internal mapping mechanism will merge the numerical value "500" with the unit "ten thousand yuan" at the ID level to generate a new virtual word element ID, representing the overall semantic meaning of "5 million yuan". This processing method strongly binds the magnitude attribute of the numerical value with the dimensional attribute of the unit in the vector space.

[0033] Next, the precision identifier is calculated and superimposed. To enable the model to perceive the "precision" of the value, the number of decimal places is crucial for financial and tax calculations. The number of digits after the decimal point in the numerical string is automatically counted. For example, for the value "100.00", there are 2 significant digits; for "0.12345", there are 5 significant digits; and for the integer "1000", there are 0 significant digits. This integer value (0, 2, 5, etc.) is mapped to a specific position encoding vector with a dimension of 64, called "precision embedding". This vector is added or concatenated element-wise with the word embedding vector (often with a dimension of 768 or 1024) of the numerical value itself. In this way, the model not only knows that the value is "100" in subsequent calculations, but also knows that it is "100.00" "accurate to the cent", thus maintaining the corresponding format specifications when outputting the results.

[0034] To verify the gain of numerical unit binding on feature representation, a test scenario was constructed, and the differences between ordinary embedding and the fusion embedding of this scheme in the vector space were compared, as shown in Table 2 below.

[0035] Table 2. Numerical Unit Joint Lexical Feature Mapping Table Paid-in capital of 10 million yuan 1000 Ten thousand yuan 0 Vector(1000) + Vector(ten thousand yuan) + Pos_Enc(0) Integer amount scenarios Tax rate: 13.00% 13.00 % 2 Vector(13.00)+Vector(%)+Pos_Enc(2) Tax rates rounded to two decimal places Exchange rate 6.8954 6.8954 None (default is 1) 4 Vector(6.8954)+Vector(Default)+Pos_Enc(4) High-precision exchange rate calculation As shown in Table 2, by generating fusion feature embedding vectors, each set of values ​​carries dimensional and precision information. After introducing this mechanism, the incidence of unit conversion errors (such as treating ten thousand yuan as yuan) in the model has been reduced from 15% to less than 0.5%, improving the accuracy of understanding financial and tax data.

[0036] S3: The fused features are embedded into the vector input to the improved Transformer encoder. In the self-attention calculation layer, a numerical semantic attention weight adjustment factor is introduced to generate a corrected attention score based on the dynamic weight calculation formula, and numerical semantic association features in the financial and tax text sequence are extracted accordingly.

[0037] When processing non-numerical and non-canonical keywords in plain text, the improved Transformer encoder automatically sets the parameters corresponding to the plain text words to zero, causing the corrected attention score to revert to the original base attention score.

[0038] The specific process of generating the corrected attention score based on the dynamic weight calculation formula uses the following formula: ; in, This represents the corrected attention score. This represents the original base attention score calculated based on the dot product of the query vector and the key vector. This represents a learnable, adjustable parameter for numerical features. The feature weights represent the numerical unit joint words in the input sequence. This represents a learnable, adjustable parameter based on the characteristics of the specification. The feature weight values ​​represent the keywords related to financial and tax regulations in the input sequence.

[0039] The learnable adjustment parameters are given preset non-zero initial values ​​during the model initialization phase and are dynamically updated during supervised adjustment as the gradient descent direction of the cross-entropy loss function changes.

[0040] The generated fusion feature embedding vector is input into the improved Transformer encoder. Traditional Transformers primarily rely on semantic associations between terms (e.g., the association between "apple" and "eat") when calculating attention scores, neglecting the magnitude logic between numerical values ​​or the strong constraints between numerical values ​​and legal terms. Therefore, a dynamic weight adjustment module is inserted into the standard Key-Query-Value calculation process. In the self-attention calculation layer, for each attention head, in addition to calculating the standard dot product attention, a numerical semantic weight adjustment factor is calculated in parallel. Specifically, when the query vector represents a field to be calculated (e.g., "tax payable"), numerical terms in the key vector (e.g., "50000", "25%)" should receive higher attention. To achieve this, the model introduces two key learnable parameters: and ,parameter Specifically designed to amplify the influence of numerical unit conjunctions, and This is used to capture the importance of key terms in tax and financial regulations (such as "deduction", "exemption", and "maximum").

[0041] The parameters are not randomly set during model initialization, but are assigned initial values ​​with physical meaning, for example, Initialized to 0.5. Initializing to 0.3 means that, in the initial state, the weights of numerical features are increased by an additional 50%. As training progresses, the parameters will be fine-tuned based on gradient feedback. In the calculation of a certain layer, numerical precision has a significant impact on the result. The value may be automatically updated to 0.8 or higher; conversely, if the current layer mainly processes text logic, It may decay.

[0042] The specific formula for calculating dynamic weights is as follows: ; The raw basic attention score is obtained by calculating the standard dot product. For example, when processing the query term "tax payable" and the keyword "25%", assume that the raw score calculated by Transformer... The initial value is 0.4. Next, we need to obtain two feature weight values: 1. (Numerical Feature Weight): This value is determined by judging whether the key term belongs to the "numerical unit joint term". If the current key term is numerical (such as "25%)", then... Activated as 1; for plain text, it is 0 or 2. (Specification Feature Weight): This value is determined by referring to a preset fiscal and tax keyword table. If the current keyword element belongs to a specification keyword (such as "tax rate"), it is activated to 1; otherwise, it is 0. Assume that in this calculation, the model has undergone partial training, and the learnable adjustment parameter has been updated to 0.6, and the parameter has been updated to 0.4. At this time, substitute the specific values into the formula for calculation: For the keyword element "25%" (which is both a numerical value and a specification word related to the tax rate, assume that here it mainly triggers the numerical feature, , ): ; It can be seen that after adjustment, the attention score has been significantly improved from 0.4 to 0.64, which means that the model will allocate more "energy" to focus on this tax rate value.

[0043] For another example, for the keyword element "according to" (a common conjunction, , ,): . The attention score of ordinary word elements remains unchanged, thus achieving "soft weighting" of key fiscal and tax information. Record the changes in the attention distribution of the model when processing complex tax calculation logics, as shown in Table 3 below.

[0044] Table 3 Experimental Data Table of Attention Weight Adjustment Effect Calculate output VAT 13% (tax rate) 0.35 0.5 0.525 Scores improved by 50%, and the model accurately captured tax rates. Calculate the additional deduction for R&D expenses 100% (deduction percentage) 0.28 0.8 0.504 The score improved significantly, avoiding the omission of percentages. Plain text description None (background text) 0.15 0.5 0.15 The score remained unchanged, and background noise was effectively filtered. As shown in Table 3, by introducing and parameters, the average attention score of key numerical values has been increased by about 60% (from the range of 0.2 - 0.3 to above 0.5), directly resulting in a significant improvement in the accuracy of the model in extracting numerical semantic association features, effectively solving the problem of numerical loss or misalignment in long texts; in addition, the improved Transformer encoder also has a "fallback mechanism". When processing ordinary text word elements that are neither numerical nor specification keywords (such as "of", "is", "in"), it is recognized that and are both 0. At this time, the correction term automatically returns to zero, and the attention calculation falls back to the standard Transformer logic, ensuring that the model's ability to understand general language will not degenerate.

[0045] S4: Based on the fiscal and tax calculation instruction fine-tuning dataset, use the cross-entropy loss function to calculate the difference between the generated result and the standard answer, and update the model parameters through backpropagation to complete the supervised adjustment of the basic large model.

[0046] The specific limitation of the calculation logic of the cross-entropy loss function is: only calculate the difference between the sequence of calculation steps and the final result value generated by the model and the standard label in the fine-tuning dataset of the financial and tax calculation instructions, and ignore the prediction loss of the input text part.

[0047] The method output includes numerical calculation results and corresponding calculation logic explanation text, which explicitly references the names of tax law clauses used as the basis for the calculation.

[0048] After feature extraction and forward computation are completed, the supervised adjustment phase begins. The goal of this phase is to optimize the model parameters using the backpropagation algorithm and the pre-built financial and tax calculation instruction fine-tuning dataset, enabling it to possess professional financial and tax calculation and reasoning capabilities. First, the design logic of the loss function is clarified. General language model training usually calculates the cross-entropy loss of all output tokens. However, in the vertical field of finance and taxation, the input part (such as "Please calculate the tax amount of Company X...") is a known condition. It is meaningless for the model to predict the input text itself, and it may even lead to overfitting. Therefore, this solution strictly limits the calculation range of the cross-entropy loss function. Specifically, a mask is created. When calculating the loss, the weights corresponding to the input text part are reset to 0, and only the difference between the "calculation step sequence" and the "final result value" part is calculated.

[0049] Assume the output sequence of the model is The standard answer sequence is The cross-entropy loss function is calculated as follows: in, It is the total number of effective predicted terms (i.e., the computation steps and the result part). It refers to the size of the vocabulary. It is a unique hot code for the real label. It is the probability distribution predicted by the model.

[0050] During training, the AdamW optimizer was used, an algorithm that performs well in fine-tuning large models. The hyperparameter settings were verified through multiple rounds of experiments: the learning rate was set to 2e-5, which is a relatively small learning rate, in order to avoid destroying the existing general knowledge of the pre-trained model; the batch size was set to 128; and the weight decay was set to 0.01. In addition, a warm-up mechanism was introduced, in which the learning rate was linearly increased from 0 to 2e-5 in the first 500 steps of training, and then gradually decreased according to the cosine annealing strategy.

[0051] The training process is divided into multiple epochs. At the end of each epoch, the model is evaluated using a validation set to ensure learnable and adjustable parameters (as mentioned above). and It can effectively update and monitor the change curve with gradient. Experiments show that in the early stages of training, The value fluctuates significantly, indicating that the model is rapidly adjusting its dependence on numerical features; approximately after the third epoch, The convergence value tends to stabilize at around 0.75, indicating that the model has found the optimal balance between numerical and semantic weights.

[0052] To verify the effectiveness of the supervised adjustment, the optimized model and the base model were compared on the "CPA Exam Calculation Questions" test set, as shown in Table 4 below.

[0053] Table 4 Comparison of Model Supervision and Adjustment Effects Basic large model (untuned) 0 32.5 4.2 1.5 This scheme optimizes the model (Epoch 1). 1000 68.4 7.5 1.6 This solution optimizes the model (Epoch 3). 3000 89.7 9.6 1.6 This solution optimizes the model (Epoch 5). 5000 91.2 9.8 1.6 As shown in Table 4, after 3 epochs (approximately 3000 steps) of targeted supervised adjustments, the model's accuracy jumped from the original 32.5% to 89.7%. More importantly, the "Computational Logic Completeness Score" reached 9.6 out of 10, meaning that the model not only calculated the correct result, but also generated a "Computational Logic Explanation Text" that clearly cited tax law provisions, with rigorous steps that fully complied with the requirements of financial and tax audit standards. For example, when processing the calculation of "taxable income for corporate income tax", the model can clearly output: "According to Article 9 of the Corporate Income Tax Law, charitable donations within 12% of the annual total profit are allowed to be deducted when calculating taxable income. In this case, the total profit is 1 million yuan, the limit is 120,000 yuan, and the actual donation is 150,000 yuan. Therefore, the taxable income is increased by 30,000 yuan." This explicit logical explanation is a direct technical effect brought about by ignoring the prediction loss of the input text and focusing on training the inference chain.

[0054] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A large-scale model optimization method for fiscal and tax numerical calculations, characterized in that, The method includes: By analyzing historical cases of corporate tax declarations and financial audits, five-dimensional features—input text, numerical values, calculation steps, correct results, and regulatory basis—are extracted to construct a fine-tuning dataset of financial and tax calculation instructions. Obtain the sequence of financial and tax texts to be processed, identify the numerical values ​​and units of measurement in the sequence of financial and tax texts through the numerical unit binding embedding unit, merge the numerical values ​​and the adjacent units of measurement into a numerical unit joint word, add a precision flag bit to the numerical unit joint word, and generate a fusion feature embedding vector. The fused features are embedded into a vector and input into an improved Transformer encoder. In the self-attention calculation layer, a numerical semantic attention weight adjustment factor is introduced to generate a corrected attention score based on a dynamic weight calculation formula, and numerical semantic association features in the financial and tax text sequence are extracted accordingly. Based on the aforementioned tax calculation instructions, the dataset is fine-tuned, and the difference between the generated results and the standard answer is calculated using the cross-entropy loss function. The model parameters are then updated through backpropagation to complete the supervised adjustment of the basic large model.

2. The large-scale model optimization method for fiscal and tax numerical calculations according to claim 1, characterized in that, The process of generating the fused feature embedding vector specifically includes: Establish a dedicated unit dictionary containing commonly used financial and tax measurement units, traverse the financial and tax text sequence, and when a numerical field is detected and is immediately followed by a unit field in the dedicated unit dictionary, map the numerical field and the unit field to a unique numerical unit joint term. Calculate the number of significant digits after the decimal point in the numerical field, map the significant digits to the corresponding positional encoding vector as the precision identifier, and superimpose the precision identifier into the word embedding vector of the numerical unit joint lexical.

3. The large-scale model optimization method for fiscal and tax numerical calculations according to claim 1, characterized in that, The specific process of generating the corrected attention score based on the dynamic weight calculation formula adopts the following formula: ； in, This represents the corrected attention score. This represents the original base attention score calculated based on the dot product of the query vector and the key vector. This represents a learnable, adjustable parameter for numerical features. The feature weight values ​​represent the numerical unit joint words in the input sequence. This represents a learnable, adjustable parameter based on the characteristics of the specification. The feature weight values ​​represent the keywords related to financial and tax regulations in the input sequence.

4. The large-scale model optimization method for fiscal and tax numerical calculations according to claim 3, characterized in that, The learnable adjustable parameters are given preset non-zero initial values ​​during the model initialization phase, and are dynamically updated during the supervised adjustment process in accordance with the gradient descent direction of the cross-entropy loss function.

5. The large-scale model optimization method for fiscal and tax numerical calculations according to claim 1, characterized in that, The process of constructing the fine-tuning dataset for financial and tax calculation instructions specifically includes: Unstructured text is extracted from the original financial and tax documents, and the amount, tax rate, and time values ​​are extracted as a set of key values ​​using preset regular expressions. Based on the provisions of current tax laws, the calculation process from the set of key values ​​to the final result is parsed into a logical chain described in natural language, which serves as the calculation steps; The corresponding tax law clause chapter code is used as the standard basis, and the input text, the numerical value, the calculation steps, the correct result and the standard basis are formatted into training samples according to the preset instruction template.

6. The large-scale model optimization method for fiscal and tax numerical calculations according to claim 1, characterized in that, The specific limitations on the calculation logic of the cross-entropy loss function are as follows: Only the differences between the sequence of calculation steps and the final result value generated by the model and the standard annotation in the fine-tuning dataset of the financial and tax calculation instructions are calculated, ignoring the prediction loss of the input text portion.

7. The large-scale model optimization method for fiscal and tax numerical calculations according to claim 1, characterized in that, When processing ordinary text lexical units that are not numerical or non-canonical keywords, the improved Transformer encoder automatically sets the parameters corresponding to the ordinary text lexical units to zero, causing the corrected attention score to revert to the original basic attention score.

8. The large-scale model optimization method for fiscal and tax numerical calculations according to claim 1, characterized in that, The method also includes a simulated sample enhancement step: Based on known financial and tax calculation formula templates, by randomly generating values ​​that conform to the business logic range and replacing the variables in the financial and tax calculation formula templates, a batch of simulated financial and tax calculation cases are automatically generated, and the simulated financial and tax calculation cases are mixed into the financial and tax calculation instruction fine-tuning dataset.

9. The large-scale model optimization method for fiscal and tax numerical calculations according to claim 1, characterized in that, The sequence of financial and tax documents includes notes to the enterprise value-added tax return, explanations to the enterprise income tax settlement report, and explanations to the financial statements.

10. The large-scale model optimization method for fiscal and tax numerical calculations according to claim 1, characterized in that, The method outputs numerical calculation results and corresponding calculation logic explanation text, in which the names of tax law clauses used as the basis for the calculation are explicitly referenced.

Images