Financial bill auditing and decision-making method and system, terminal and medium

Abstract

The invention relates to the field of bill auditing, and particularly provides a financial bill auditing decision-making method and system, a terminal and a medium, and the method comprises the steps: collecting multi-mode finance and tax data including texts, images and structured data, and carrying out the preprocessing and alignment; carrying out feature extraction on the preprocessed data by using a multi-modal large model, carrying out feature fusion by using a dynamic weight distribution algorithm based on the credibility of each modal, the service priority and historical feedback, and obtaining a risk probability through a risk identification neural network; the risk probability is compared with a preset threshold value and rule in an auditing rule base, automatic passing is triggered, after risk abnormity is recorded, passing is conducted, and auditing actions such as manual auditing or starting of a high-risk emergency plan are pushed; and finally, adjusting model weight parameters according to manual feedback information to realize system self-optimization. The auditing decision-making efficiency is improved, and the accuracy and the service adaptability are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of bill auditing, in particular to a financial bill auditing decision method, system, terminal and medium. BACKGROUND

[0002] The financial and tax bill auditing business naturally involves multi-modal data, including text, images and structured data. Most traditional technical solutions independently process and analyze single-modal data. For example, some systems focus on OCR recognition of invoice images, and other systems process structured reimbursement single data, lacking a unified framework to associate these heterogeneous data for cross-validation, resulting in a serious dependence on manual checking between different systems, low efficiency and easy errors. Traditional automated auditing systems usually hardcode or configure business rules in the system. When business rules need to be adjusted, the code or configuration must be modified by technical personnel and the system must be redeployed, which cannot be quickly responded by business personnel. In addition, these rule engines can usually only handle simple logical judgments and are difficult to handle complex and comprehensive business scenarios. SUMMARY

[0003] To solve the above problems, the present application provides a financial bill auditing decision method, system, terminal and medium, which improves the auditing decision efficiency, improves the accuracy and business adaptability.

[0004] In a first aspect, the technical solution of the present application provides a financial bill auditing decision method, comprising the following steps: Collecting multi-modal data associated with the financial and tax bill business to be audited, and establishing a unified association identifier for the multi-modal data, the multi-modal data including text data, image data and structured data; Preprocessing the collected multi-modal data to generate standardized text feature data, image feature data and structured feature data; Inputting the preprocessed multi-modal data into a multi-modal large model for processing, including: using an encoder corresponding to the modal to extract a feature vector of the corresponding feature data, to obtain a text feature vector, an image feature vector and a structured feature vector; using a dynamic weight distribution algorithm to weight and fuse each type of feature vector to generate a fused feature vector; inputting the fused feature vector into a risk identification neural network to obtain a risk probability of the financial and tax bill to be audited; wherein the dynamic weight distribution algorithm dynamically calculates and distributes the fusion weight of each modal based on the credibility of each modal data, the pre-defined business risk priority and the historical auditing feedback result; The risk probability is compared with a preset risk threshold in an audit rule library, and one or more of the following is triggered according to a comparison result: automatic passing, passing after recording a risk exception, pushing to manual audit, and triggering a high-risk emergency plan; Artificial feedback information for the audit result is received, and weight parameters in the multi-modal large model are adjusted based on the artificial feedback information.

[0005] In a second aspect, the technical solution of the present application provides a financial document auditing decision system, comprising: A multi-modal data acquisition module is configured to acquire multi-modal data associated with a financial tax document to be audited, and establish a unified association identifier for the multi-modal data, wherein the multi-modal data includes text data, image data, and structured data. A multi-modal data preprocessing module is configured to preprocess the acquired multi-modal data to generate standardized text feature data, image feature data, and structured feature data. A risk probability generation module is configured to input the preprocessed multi-modal data into a multi-modal large model for processing, including: using an encoder corresponding to each modality to extract a feature vector of the corresponding feature data to obtain a text feature vector, an image feature vector, and a structured feature vector; using a dynamic weight distribution algorithm to weight and fuse each type of feature vector to generate a fused feature vector; and inputting the fused feature vector into a risk identification neural network to obtain a risk probability of the financial tax document to be audited; wherein the dynamic weight distribution algorithm dynamically calculates and distributes the fusion weights of each modality based on the credibility of each modality data, a predefined business risk priority, and a historical audit feedback result. An audit action triggering module is configured to compare the risk probability with a preset risk threshold in an audit rule library, and trigger a corresponding audit action according to a comparison result, including one or more of the following: automatic passing, passing after recording a risk exception, pushing to manual audit, and triggering a high-risk emergency plan. A large model optimization adjustment module is configured to receive artificial feedback information for the audit result, and adjust weight parameters in the multi-modal large model based on the artificial feedback information.

[0006] In a third aspect, the technical solution of the present application provides a terminal, comprising: A memory is configured to store a financial document auditing decision program. A processor is configured to execute the financial document auditing decision program to implement the steps of the financial document auditing decision method according to any one of the above.

[0007] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a financial bill auditing and decision program, and the financial bill auditing and decision program, when executed by a processor, implements the steps of the financial bill auditing and decision method according to any one of the preceding aspects.

[0008] From the above technical solutions, the present application has the following advantages: by constructing a unified processing framework based on a multi-modal large model, the present application can automatically collect, align and fuse key information from different modalities, implement cross-validation, reduce the dependence of manual checking between different systems, and improve the auditing efficiency and accuracy; by placing the auditing logic behind the quantitative risk probability output by the large model and combining it with a configurable rule base, the financial personnel can directly adjust the mapping relationship between the risk threshold and the auditing action according to the business requirements through a visual interface, without modifying the underlying code or restarting the system, thereby enhancing the response capability; by utilizing the feature extraction and semantic understanding capabilities of the multi-modal large model, the present application can comprehensively integrate the features and correlations of multi-source heterogeneous data, and adapt to complex scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0010] Figure 1 A financial bill auditing and decision method flowchart provided by an embodiment of the present application.

[0011] Figure 2 A financial bill auditing and decision system structure schematic diagram provided by an embodiment of the present application.

[0012] Figure 3 A terminal structure schematic diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to make the application purposes, features and advantages of the present application more obvious and easy to understand, the following will use specific embodiments and drawings to clearly and completely describe the technical solutions protected by the present application. Obviously, the following described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0014] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0015] Figure 1 A flowchart of a financial bill auditing decision method provided by an embodiment of the present application is shown. In the flowchart, Figure 1 The execution subject can be a financial bill auditing decision system. The financial bill auditing decision method provided by the embodiment of the present application is executed by a computer device, and accordingly, the financial bill auditing decision system runs in the computer device. According to different needs, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0016] As Figure 1 shown, the method comprises the following steps.

[0017] S1, collecting multi-modal data associated with a to-be-audited financial tax bill business, and establishing a unified association identifier for the multi-modal data, wherein the multi-modal data comprises text data, image data and structured data; S2, preprocessing the collected multi-modal data to generate standardized text feature data, image feature data and structured feature data.

[0018] S3, inputting the preprocessed multi-modal data into a multi-modal large model for processing, comprising: extracting feature vectors of corresponding feature data by using an encoder corresponding to the modal, to obtain text feature vectors, image feature vectors and structured feature vectors; using a dynamic weight distribution algorithm to weight and fuse each type of feature vector to generate a fused feature vector; inputting the fused feature vector into a risk identification neural network to obtain a risk probability of the to-be-audited financial tax bill; wherein the dynamic weight distribution algorithm dynamically calculates and distributes the fusion weight of each modal based on the credibility of each modal data, the pre-defined business risk priority and the historical audit feedback result.

[0019] S4, comparing the risk probability with a pre-set risk threshold in an audit rule library, triggering one or more of the following according to the comparison result: automatic passing, passing after recording risk abnormalities, pushing to manual auditing, triggering a high-risk emergency plan.

[0020] S5, receiving manual feedback information for the auditing result, and adjusting the weight parameters in the multi-modal large model based on the manual feedback information.

[0021] Further, as a refinement and extension of the above embodiment, in order to fully describe the specific implementation process in this embodiment, another financial bill auditing decision method is provided, which comprises the following steps.

[0022] S101, a multi-modal large model is constructed and trained.

[0023] The multi-modal large model of the embodiment is a deep neural network, which comprises a feature encoding layer, a feature fusion layer and a risk identification layer, and is used to convert multi-modal input into a risk probability.

[0024] The feature encoding layer comprises three independent modal encoders for converting standardized feature data into high-dimensional feature vectors, which are a text encoder, an image encoder and a structured data encoder. The text encoder adopts a pre-trained model based on the Transformer architecture, which can be BERT, and the input is standardized text feature data. The text feature vector of a fixed dimension is output by encoding through a multi-layer self-attention mechanism. The image encoder adopts a visual Transformer model (ViT), and the input is standardized image feature data. The model converts the image into an image block sequence, and after linear projection and position encoding, the global visual features are extracted through the Transformer layer to output the image feature vector. The structured data encoder adopts a multi-layer perceptron (MLP), and the input is standardized structured feature data. The structured data is mapped to a feature space of the same dimension as other modalities through a plurality of fully connected layers and activation functions, and the structured feature vector is output.

[0025] The feature fusion layer is used for adaptive weighted fusion of the feature vectors of the three modalities. The input is the text feature vector, the image feature vector and the structured feature vector from the feature encoding layer. According to the comprehensive credibility of each modal data, the pre-defined business risk priority and the historical feedback, the fusion weight of each modality is dynamically calculated and output through the fusion algorithm. Then, the three feature vectors are weighted and summed according to the dynamic fusion weight to generate the final fusion feature vector. Details will be introduced later, and will not be repeated here.

[0026] The risk identification layer is a multi-layer perceptron classifier, which is used for final risk assessment based on the fusion features. It contains at least one fully connected hidden layer, and uses the Sigmoid function as the activation function of the output layer. The input is the fusion feature vector output by the feature fusion layer. The fusion feature vector is nonlinearly transformed by the first fully connected layer to extract high-order risk features; then the feature dimension is mapped to 1 dimension by the second fully connected layer, and finally the output value is compressed to the interval (0, 1) by the Sigmoid function to obtain the risk probability.

[0027] The constructed multi-modal large model is trained through supervised learning to optimize all weight parameters in feature encoding, fusion and risk identification, and to realize model training, including the following steps.

[0028] Step 1, collect a large number of historical financial and tax bill case data to form a training data set.

[0029] Each sample contains multi-modal original data of a complete case, and each case is determined to have a true risk label after being audited. The label is quantified according to the risk level and used as a regression target.

[0030] Step 2, after the multi-modal data of the training sample is sequentially preprocessed, it is input into the multi-modal large model to be trained, sequentially passes through the feature encoding layer, the feature fusion layer and the risk identification layer, and finally calculates the risk probability predicted by the model. The data preprocessing is described in detail later, and will not be repeated here.

[0031] Step 3, the difference between the model prediction value and the true risk label is calculated using the binary cross-entropy loss function, and the loss function is defined as,

[0032] wherein, is the prediction value, i.e., the predicted risk probability output in step 2, is the true risk label.

[0033] For batch training, the average value of the loss of all samples in a batch is calculated.

[0034] Step 4, back propagation and parameter update.

[0035] Using the back propagation algorithm, starting from the output end of the loss function, the loss is calculated layer by layer along the model calculation graph to obtain the gradient of all trainable parameters in the model, including the weights and biases of the encoders, the fusion layer dynamic weight calculation unit and the risk identification MLP.

[0036] Using the gradient descent optimization algorithm (such as the Adam optimizer), the model parameters are iteratively updated according to the calculated gradient. Steps 2 and 4 are repeated to traverse the training data set for multiple rounds until the model loss function converges to a predetermined threshold or the performance on the validation set no longer improves significantly, completing the initial training of the model.

[0037] S102, collect multi-modal data associated with the financial and tax bill business to be audited, and establish a unified association identifier for the multi-modal data.

[0038] The multi-modal data includes text data, image data and structured data.

[0039] Text data: interface with OA system through HTTP API, parse reimbursement single JSON, JSON fields include reimburse_id "reimbursement unique identifier", emp_name "employee name", dept "department", amount "reimbursement amount", travel_date "business trip date".

[0040] Image data: synchronize OA uploaded invoice images through FTP, naming rule "reimburse_id_file type.format", store path record to MySQL index table data_collect_xx.

[0041] Structured data: synchronize ERP system financial accounts through JDBC every 10 minutes, call bank API to obtain transaction data. The fields of ERP system financial accounts include emp_id "employee ID" and remaining_budget "monthly remaining budget", and the fields of transaction data include trans_amount "transaction amount" and payee "payee".

[0042] S103, preprocessing multi-modal data.

[0043] The collected multi-modal data is preprocessed, including data cleaning, format standardization, key entity extraction and anomaly detection, and the multi-modal data is aligned, and finally the standardized text feature data, image feature data and structured feature data are generated.

[0044] S103.1, data cleaning and format standardization.

[0045] For text data, remove meaningless special characters, unify date format, and unify amount format.

[0046] For image data, Gaussian filter denoising, image tilt correction based on Hough transform, and unify image resolution to preset standard.

[0047] For structured data, handle null values, unify date and amount formats, and ensure data type correctness.

[0048] S103.2, key entity extraction and confidence output.

[0049] This step calls the corresponding recognition model to extract the key entities required for business audit from each modality data. These key entities are target entities, and the confidence output of the model is also obtained simultaneously.

[0050] For the text modality, the cleaned text data is input into the named entity recognition model to extract key entities, including reimbursement amount, business trip date, department name, and possibly project code. The named entity recognition model uses a BERT fine-tuned NER model, which outputs a recognition confidence for each successfully recognized entity. The NER model has a Softmax layer at the end of the sequence tagging, which calculates the probability of each token belonging to each entity label. The probabilities are then aggregated by taking the average to obtain the confidence of the entire entity composed of consecutive tokens.

[0051] For the image modality, the preprocessed image is input into the OCR engine to extract key entities, including invoice amount, invoice date, seller name, and stamp clarity. The OCR engine outputs a recognition confidence for each successfully recognized entity, which represents the OCR engine's degree of certainty about the entity recognition result. The OCR engine generates a confidence for each recognized text field, i.e., entity, based on the probability output of its internal classification or sequence model while performing character recognition.

[0052] For the structured data modality, predefined data validation rules are called to perform logical validation. For example, "budget balance data consistency," the confidence is calculated based on the ratio or difference between the budget amount and the reimbursement amount using a pre-defined mapping function: budget consistency confidence = max(0, min(1, (absolute value of the difference between the reimbursement amount in the reimbursement form and the current available budget balance / pre-set tolerance threshold))), which means when the reimbursement amount equals the budget balance, the difference is 0, and the confidence is 1. When the reimbursement amount exceeds the budget balance, but the excess is within the tolerance threshold, the confidence decreases linearly from 1 to a value greater than 0. When the reimbursement amount exceeds the budget balance by an amount equal to or greater than the tolerance threshold, the confidence decreases to 0. For example, "bank flow matching status," the confidence is calculated based on the similarity of the amount, time, and payee name in the flow record using cosine similarity.

[0053] S103.3, multi-modal data alignment.

[0054] Using the unique identifier of the financial and tax document as the association key, the data records of the text, image, and structured data modalities, after cleaning and entity extraction, are associated to generate a unified data alignment table.

[0055] S103.4, generate standardized feature data.

[0056] The data processed in the above steps and aligned is organized into a standardized structure, including key entities of each modality itself, and also accompanied by the confidence of each key entity generated in the extraction and verification process, for subsequent calculation of comprehensive confidence.

[0057] S104, inputting the pre-processed multi-modal data into a multi-modal large model for processing and outputting a risk probability.

[0058] The processing process of the multi-modal large model on the multi-modal data includes: using the encoder corresponding to the modality to extract the feature vector of the corresponding feature data, to obtain the text feature vector, the image feature vector and the structured feature vector; using a dynamic weight distribution algorithm to weight and fuse each type of feature vector to generate a fused feature vector; inputting the fused feature vector into a risk identification neural network to obtain the risk probability of the financial and tax document to be audited.

[0059] Wherein, the process of extracting the feature vector using the encoder is described in the related content in step S101, which will not be repeated here.

[0060] After obtaining the feature vector, a dynamic weight distribution algorithm is used to weight and fuse each type of feature vector to generate a fused feature vector. The dynamic weight distribution algorithm dynamically calculates and distributes the fusion weight of each modality based on the credibility of each modality data, the pre-defined business risk priority and the historical audit feedback result, which specifically includes the following steps.

[0061] S104.1, preset a basic weight for each modality based on the pre-defined business risk priority.

[0062] S104.11, based on the pre-defined rules related to the financial and tax document audit, generate an evaluation vector for the text, image and structured data modalities respectively.

[0063] The evaluation vector is used to quantify the business risk priority of each modality, which at least includes the accuracy key degree, the tampering difficulty and the historical error rate. The accuracy key degree represents the decisive degree of the modality data on the final audit conclusion; the tampering difficulty represents the degree of difficulty of unauthorized modification of the modality data; and the historical error rate represents the statistical error rate of the modality data in historical audits.

[0064] In the initial stage, historical audit data and its final manual audit results are collected, and by analyzing these data, the contribution of each modality in correctly identifying risks is quantified.

[0065] Accuracy key degree The score of the accuracy key degree is calculated as the proportion of the audit result consistent with the true label when the modality data is determined to be "highly credible", which is represented as,

[0066] That is, the molecule refers to the modality The total number of cases that are determined by the system as "highly reliable" and the final audit result is correct, and the denominator is the modality The total number of cases determined by the system as "highly reliable".

[0067] For example, in 1000 cases of clear images, the system judges correctly in 980 times, and the image modality . Among them, wherein , Text modality, image modality and structured data modality respectively.

[0068] Historical error rate The score is the error rate of the identification result of the modality data, and is converted into a percentage score, which is expressed as

[0069] Among them, The modality refers to The statistical error rate in the historical audit is calculated as follows: In the statistical period, the number of cases that the modality provides data or judgments that are finally confirmed to be wrong or lead to wrong decisions accounts for the proportion of the total number of cases that the modality is used or referred to by the system.

[0070] Historical error rate It is a reverse score, the lower the error rate, the higher the score , indicating that the modality has better historical performance and is more reliable.

[0071] Tampering difficulty The calculation of the score combines factors such as the system authentication level of the data source, the integrity of the data log, etc., and sets a fixed classification score table. For example: direct data interface source = 95 points, encrypted third-party transmission = 80 points, user uploaded file = 60 points.

[0072] The above dimensions of each modality are scored in percentage, forming an evaluation vector, expressed as .

[0073] S104.12, based on a preset dimension weight vector, the evaluation vector of each modality is weighted and linearly combined to calculate the risk priority score of each modality; wherein each weight component in the dimension weight vector corresponds to an evaluation dimension, and the sum of each weight component is 1.

[0074] Using the weighted summation method, the scores of each dimension are aggregated to calculate the comprehensive risk priority score of each modality, expressed as

[0075] wherein, 、 、 are the weight coefficients of each dimension, respectively.

[0076] The accuracy criticality weight reflects the enterprise's requirement for the correctness of the audit result, which is generally given the highest or higher priority.

[0077] The tampering difficulty weight reflects the enterprise's requirement for data authenticity and fraud prevention capability.

[0078] The historical error rate weight reflects the enterprise's requirement for system stability and predictability.

[0079] The specific value of the weight is pre-defined based on expert experience in the initial stage, and then optimized based on data, that is, after running for a period of time, the optimal weight combination can be found based on cross-validation of grid search, and the optimization goal is to maximize the comprehensive performance index of the entire intelligent audit system, for example, to minimize the overall false negative loss. Divide the historical data set into a training set and a validation set, run the system on the training set using different weight combinations, then evaluate the audit performance on the validation set, and finally select the set of weights that optimizes the performance index as the formal configuration.

[0080] S104.13, normalize the risk priority, including calculating the ratio of the risk priority score of each modality to the sum of the risk priority scores of all modalities, and the obtained ratio is the preset basic weight of the modality, and the sum of the preset basic weights of all modalities is 1.

[0081] The risk priority scores of each modality are normalized to obtain the preset basic weights of each modality, which are represented as:

[0082] S104.2, set a credibility threshold for each modality, which is configured based on the current historical audit feedback results.

[0083] S104.21, collect all audit case data generated within the running period or within the preset historical period, wherein each data record at least contains the modality type, the comprehensive credibility output by the modality when processing the corresponding case, and the case audit action.

[0084] Collect data of all audit cases, and the data set contains case unique identifier, modality type, comprehensive credibility, and audit action.

[0085] S104.22, from the full amount of audit case data, filter out the cases whose audit action is to push to manual audit to constitute a strong label data set.

[0086] S104.23, for each modality in the strong label dataset, divide it into different confidence intervals according to the size of its comprehensive confidence; for each confidence interval, calculate the false positive rate of the modality in the interval, which is the ratio of the number of false positive cases in the interval to the total number of cases.

[0087] Divide the confidence interval [0, 1] into H equal width intervals, for the hth interval , calculate the false positive rate of the data in the interval , expressed as

[0088] wherein, is the total number of cases in which the comprehensive confidence of the modality falls within the interval , and is the number of cases in which the case is determined to be false positive by manual review.

[0089] S104.24, according to the preset acceptable maximum false positive rate, traverse each confidence interval, and configure the lower limit value of the smallest confidence interval that meets the condition that the statistical false positive rate is not greater than the acceptable maximum false positive rate as the confidence threshold of the modality.

[0090] According to the business requirements, set a system acceptable maximum false positive rate upper limit , the confidence threshold of the modality is configured as:

[0091] That is, the boundary value of the smallest confidence interval that meets the condition that the statistical false positive rate is not greater than the acceptable maximum false positive rate is configured as the updated confidence threshold of the modality, wherein the boundary value refers to the minimum value of the confidence interval.

[0092] S104.3, calculate the comprehensive confidence of each modality in the current review case, which is the aggregation of the confidence of each target recognition item in the modality.

[0093] S104.31, for each modality in the current review case, obtain the confidence output by the corresponding recognition model after processing the internal target recognition item in the pre-processing process; the target recognition item of the image modality includes invoice amount, invoice date, seller name, and seal clarity, and the recognition model is an OCR engine; the target recognition item of the text modality includes reimbursement amount, business trip date, and department name, and the recognition model is a named entity recognition model; the target recognition item of the structured data modality includes budget balance data consistency and bank flow matching state, and the recognition model is a data verification rule.

[0094] ​The calculation of the confidence level is detailed in the preprocessing process of step S103, which will not be repeated here.

[0095] S104.32, for each modality, based on the business importance of each target identification item in the financial and tax audit, a weight is assigned to it.

[0096] In an optional embodiment, the weight assignment is realized based on the structured implementation of the analytic hierarchy process. First, invite several experts in the field of finance and risk control, and for all target identification items under the same modality, according to the "1-9 scale method", make pairwise importance comparison to construct a judgment matrix. Calculate the characteristic vector of the judgment matrix and perform consistency check. If passed, the normalized weight vector can be obtained.

[0097] S104.33, the confidence levels of the target identification items are weighted and averaged to generate the comprehensive confidence level of the modality.

[0098] S104.4, compare the comprehensive confidence of each modality with its corresponding confidence threshold, calculate the confidence deviation coefficient, which is used to quantify the reliability of the modality data, and is expressed as

[0099] wherein, is the confidence deviation coefficient, is the comprehensive confidence, is the confidence threshold, wherein , represents the text modality, represents the image modality, represents the structured data modality.

[0100] S104.5, use the calculated confidence deviation coefficient to modify the basic weight to obtain the final dynamic fusion weight.

[0101] The basic weight is modified by the following formula to obtain the final dynamic fusion weight , which is expressed as

[0102] S104.6, according to the dynamic fusion weight, the feature vectors of each modality are weighted and summed to generate the final fusion feature vector.

[0103] The final fusion feature vector is input into the risk identification neural network to obtain the risk probability of the financial and tax bill to be audited.

[0104] S105, according to the risk probability, match the audit action.

[0105] The risk probability is compared with a preset risk threshold in the audit rule library, and one or more of the following is triggered according to the comparison result: automatic passing, passing after recording risk abnormalities, pushing to manual audit, and triggering a high-risk emergency plan.

[0106] Specifically, the risk probability is matched with a predefined risk threshold interval in the audit rule library, and is mapped to a corresponding risk level; the risk threshold interval includes: a first interval [0, 0.2), which is mapped to a no significant risk level, a second interval [0.2, 0.5), which is mapped to a low risk level, a third interval [0.5, 0.7), which is mapped to a medium risk level, and a fourth interval [0.7, 1], which is mapped to a high risk level; according to the risk level obtained by mapping, the audit action preset in the audit rule library corresponding to the risk level is triggered; wherein the no significant risk level triggers the automatic passing action, the low risk level triggers the passing after recording risk abnormalities action, the medium risk level triggers the pushing to manual audit action, and the high risk level triggers the starting high risk emergency plan action.

[0107] S106, model optimization.

[0108] Receiving manual feedback information for the audit result, adjusting the weight parameters in the multi-modal large model based on the manual feedback information, specifically including the following steps.

[0109] S106.1, structurally process the received manual feedback information, and generate a standardized feedback record for each misjudgment case, which includes a unique case identifier, a correct risk level confirmed by manual review, and a real risk probability mapped based on the correct risk level.

[0110] S106.2, according to the unique case identifier, call the multi-modal data corresponding to the case, and input it into the multi-modal large model again to perform forward propagation calculation to obtain the predicted risk probability of the multi-modal large model for the case.

[0111] S106.3, based on the real risk probability and the predicted risk probability of the current case, calculate the loss value of the multi-modal large model on the current case through the cross-entropy loss function.

[0112] S106.4, using the back propagation algorithm, starting from the output end of the cross-entropy loss function, calculating the gradient of the loss value to at least one trainable weight parameter matrix in the multi-modal large model layer by layer along the model calculation graph; the trainable weight parameter matrix includes the weight in the feature fusion network and the risk identification neural network.

[0113] S106.5, according to the calculated gradient and the preset learning rate, iteratively updating the trainable weight parameter matrix through the gradient descent method.

[0114] Specifically, the financial staff marks the "misjudgment cases" after reviewing on the Web end, containing 3 types of core information: Basic information: reimburse_id (reimbursement ID), audit_id (audit record ID), feedback_time (feedback time); Misjudgment type: preset enumeration value; Correct label: corrected risk level.

[0115] Only "high credibility feedback" is retained, which requires cross-confirmation of misjudgment results by 2 financial supervisors and can locate specific risk characteristics, and invalid feedback of "operation error" and "incomplete information" is excluded.

[0116] Through the association of the original multi-modal features of the reimbursement ID, a "feedback-feature mapping table" is generated to clearly indicate the core impact feature dimensions corresponding to each feedback.

[0117] The mapping relationship of "misjudgment type - weight dimension" is preset, the dimension of the large model is located according to the misjudgment type of the user feedback, and the weight of the dimension is adjusted based on the gradient descent fine-tuning method.

[0118] The above describes an embodiment of a financial bill auditing and decision-making method in detail, and based on the financial bill auditing and decision-making method described above, the embodiment of the present application further provides a financial bill auditing and decision-making system corresponding to the method.

[0119] Figure 2 A financial bill auditing and decision-making system structure schematic diagram is provided in the embodiment of the present application, in the embodiment, the financial bill auditing and decision-making system 200 can be divided into multiple functional modules according to the functions it performs, as shown in the figure. The module referred to in the present application refers to a series of computer program segments that can be executed by at least one processor and can complete a fixed function, which is stored in the memory. Figure 2

[0120] The multi-modal data acquisition module 210 is used to acquire multi-modal data associated with the business of the financial tax bill to be audited, and establish a unified association identifier for the multi-modal data, the multi-modal data including text data, image data and structured data.

[0121] The multi-modal data preprocessing module 220 is used to preprocess the acquired multi-modal data to generate standardized text feature data, image feature data and structured feature data.

[0122] ​The risk probability generation module 230 is configured to input the preprocessed multi-modal data into the multi-modal large model for processing, including: extracting feature vectors of corresponding feature data by using an encoder corresponding to a mode to obtain a text feature vector, an image feature vector, and a structured feature vector; performing weighted fusion on the feature vectors of various types by using a dynamic weight distribution algorithm to generate a fused feature vector; and inputting the fused feature vector into a risk identification neural network to obtain a risk probability of the financial tax document to be audited; wherein the dynamic weight distribution algorithm dynamically calculates and distributes the fusion weights of the modes based on the credibility of the multi-modal data, a pre-defined business risk priority, and a historical audit feedback result.

[0123] The audit action triggering module 240 is configured to compare the risk probability with a pre-set risk threshold in an audit rule library, and trigger one or more of automatic passing, passing after recording a risk exception, pushing to manual audit, and triggering a high-risk emergency plan according to a comparison result.

[0124] The large model optimization and adjustment module 250 is configured to receive manual feedback information for an audit result, and adjust weight parameters in the multi-modal large model based on the manual feedback information.

[0125] The financial document auditing decision system of the embodiment is used to implement the financial document auditing decision method described above, and therefore the specific implementation manners in the system can be seen from the embodiment part of the financial document auditing decision method described above, and therefore the specific implementation manners can be referred to the description of the corresponding embodiment part, and will not be introduced here.

[0126] In addition, since the financial document auditing decision system of the embodiment is used to implement the financial document auditing decision method described above, the functions thereof correspond to the functions of the method described above, and will not be described here.

[0127] Figure 3 A structure schematic diagram of a terminal 300 provided by the embodiment of the present application includes a processor 310, a memory 320, and a communication unit 330. The processor 310 is configured to implement the process steps of the financial document auditing decision method embodiment described above when implementing the financial document auditing decision program stored in the memory 320.

[0128] The present application also provides a computer storage medium. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc. The computer storage medium stores a financial document auditing decision program. When the financial document auditing decision program is executed by a processor, the process steps of the financial document auditing decision method embodiment described above are implemented.

[0129] The foregoing description of the disclosed embodiments enables one skilled in the art to make or use the application. Numerous modifications of those embodiments can be apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without the use of the innovation falling outside the spirit and scope of the application. Therefore, the application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for making decisions on the review of financial documents, characterized in that, Includes the following steps: Collect multimodal data related to the pending financial and tax invoices business, and establish a unified association identifier for the multimodal data, which includes text data, image data, and structured data; The collected multimodal data is preprocessed to generate standardized text feature data, image feature data, and structured feature data; The preprocessed multimodal data is input into a multimodal large model for processing, including: extracting feature vectors of corresponding feature data using encoders corresponding to the modalities to obtain text feature vectors, image feature vectors, and structured feature vectors; using a dynamic weight allocation algorithm to weight and fuse various feature vectors to generate a fused feature vector; and inputting the fused feature vector into a risk identification neural network to obtain the risk probability of the financial and tax invoices to be audited. Among these, the dynamic weight allocation algorithm dynamically calculates and allocates the fusion weights of each modality based on the credibility of each modality's data, predefined business risk priorities, and historical audit feedback results. The risk probability is compared with the preset risk threshold in the audit rule base, and corresponding audit actions are triggered based on the comparison results, including one or more of the following: automatic approval, approval after recording risk anomalies, push to manual review, and triggering high-risk emergency plans. Receive manual feedback on the review results and adjust the weight parameters in the multimodal large model based on the manual feedback.

2. The method according to claim 1, characterized in that, A dynamic weight allocation algorithm is used to weight and fuse various feature vectors to generate a fused feature vector, specifically including: Each modality is assigned a basic weight based on a predefined business risk priority. Set a credibility threshold for each modality, which is configured based on the current historical audit feedback results; Calculate the overall credibility of each modality in the current audit case. This overall credibility is the aggregation of the confidence scores of each target identification item within the modality. The overall reliability of each modality is compared with its corresponding reliability threshold, and a reliability deviation coefficient is calculated to quantify the reliability of the modality data. This coefficient is expressed as: in, This is the credibility deviation coefficient. To assess overall credibility, , where is the credibility threshold. , Represents text modality, Represents image modality, Represents structured data modes; Using the calculated confidence deviation coefficient The base weights are adjusted to obtain the final dynamic fusion weights; The feature vectors of each modality are weighted and summed according to the dynamic fusion weights to generate the final fused feature vector.

3. The method according to claim 2, characterized in that, Each modality is assigned a base weight based on predefined business risk priorities, specifically including: Based on predefined rules related to the review of financial and tax invoices, an evaluation vector is generated for text, image, and structured data modalities respectively. The evaluation vector is used to quantify the business risk priority of each modality, and its dimensions include at least accuracy criticality, tampering difficulty, and historical error rate: accuracy criticality indicates the degree of decisiveness of the modality data for the final review conclusion; tampering difficulty indicates the difficulty of the modality data being modified without authorization; historical error rate indicates the statistical error rate of the modality data in historical reviews. Based on a preset dimension weight vector, the evaluation vector of each modality is weighted and linearly combined to calculate the risk priority score of each modality; wherein, each weight component in the dimension weight vector corresponds to an evaluation dimension, and the sum of all weight components is 1. The risk priority is normalized by calculating the ratio of the risk priority score of each modality to the sum of the risk priority scores of all modalities. The resulting ratio is the preset basic weight of that modality, and the sum of the preset basic weights of all modalities is 1.

4. The method according to claim 2, characterized in that, A confidence threshold is set for each modality, specifically including: Collect all audit case data generated within the operating cycle or within a preset historical period. Each data record must include at least the modality type, the overall credibility output by the modality when processing the corresponding case, and the audit action for that case. From the full volume of reviewed cases, cases whose review action was to push them to human review were selected to form a strongly labeled dataset; For each modality in the strongly labeled dataset, it is divided into different confidence intervals based on its overall confidence level. For each confidence interval, the misclassification rate of that modality within that interval is calculated. The misclassification rate is the ratio of the number of misclassified cases within the interval to the total number of cases. Based on the preset acceptable maximum false positive rate, it iterates through each confidence interval and sets the lower limit of the minimum confidence interval that satisfies the condition that the statistical false positive rate is not greater than the acceptable maximum false positive rate as the confidence threshold of that modality.

5. The method according to claim 2, characterized in that, Calculate the overall credibility of each modality in the current audit case, specifically including: For each modality in the current audit case, obtain the confidence level of its internal target recognition items after processing by the corresponding recognition model during the preprocessing process; the target recognition items for the image modality include invoice amount, invoice date, seller name, and seal clarity, and the recognition model is an OCR engine; the target recognition items for the text modality include reimbursement amount, business trip date, and department name, and the recognition model is a named entity recognition model; the target recognition items for the structured data modality include budget balance data consistency and bank statement matching status, and the recognition model is data verification rules. For each modality, weights are assigned to each target identification item based on its business importance in the financial and tax audit. The overall confidence level of the modality is generated by weighted averaging of the confidence levels of each target identification item.

6. The method according to claim 1, characterized in that, The risk probability is compared with a preset risk threshold in the audit rule base, and a corresponding audit action is triggered based on the comparison result, specifically including: The risk probability is matched with the predefined risk threshold range in the audit rule base and mapped to the corresponding risk level; the risk threshold range includes: the first range [0, 0.2), mapped to no significant risk level; the second range [0.2, 0.5), mapped to low risk level; the third range [0.5, 0.7), mapped to medium risk level; and the fourth range [0.7, 1], mapped to high risk level. Based on the risk level obtained from the mapping, the pre-set review action in the review rule base that uniquely corresponds to the risk level is triggered. Among them, no significant risk level triggers the action of automatic approval, low risk level triggers the action of recording risk anomalies and then approval, medium risk level triggers the action of pushing to manual review, and high risk level triggers the action of activating the high risk emergency plan.

7. The method according to claim 1, characterized in that, Receive manual feedback on the review results, and adjust the weight parameters in the multimodal large model based on the manual feedback, specifically including: The received human feedback information is structured and a standardized feedback record is generated for each misjudgment case. The record includes a unique case identifier, the correct risk level confirmed by human review, and the actual risk probability mapped based on the correct risk level. Based on the unique case identifier, retrieve the corresponding multimodal data for that case, and input it again into the multimodal large model to perform forward propagation calculation, thereby obtaining the current predicted risk probability of the multimodal large model for that case; Based on the true risk probability and the current predicted risk probability for this case, the loss value of the multimodal large model on the current case is calculated using the cross-entropy loss function; Using the backpropagation algorithm, starting from the output of the cross-entropy loss function, the gradient of the loss value with respect to at least one trainable weight parameter matrix in the multimodal large model is calculated layer by layer in reverse along the model computation graph; the trainable weight parameter matrix includes the weights in the feature fusion network and the risk recognition neural network. Based on the calculated gradient and the preset learning rate, the trainable weight parameter matrix is ​​iteratively updated using the gradient descent method.

8. A financial invoice review and decision-making system, characterized in that, include: The multimodal data acquisition module is used to collect multimodal data related to the financial and tax invoices to be reviewed, and to establish a unified association identifier for the multimodal data. The multimodal data includes text data, image data, and structured data. The multimodal data preprocessing module is used to preprocess the collected multimodal data to generate standardized text feature data, image feature data, and structured feature data; The risk probability generation module is used to input preprocessed multimodal data into a multimodal large model for processing. This includes: extracting feature vectors of corresponding feature data using encoders corresponding to each modality to obtain text feature vectors, image feature vectors, and structured feature vectors; using a dynamic weight allocation algorithm to weight and fuse various feature vectors to generate a fused feature vector; and inputting the fused feature vector into a risk identification neural network to obtain the risk probability of the financial and tax invoices to be audited. The dynamic weight allocation algorithm dynamically calculates and allocates the fusion weights of each modality based on the credibility of each modality's data, predefined business risk priorities, and historical audit feedback results. The audit action triggering module is used to compare the risk probability with the preset risk threshold in the audit rule base, and trigger the corresponding audit action according to the comparison result, including one or more of the following: automatic approval, approval after recording risk anomalies, push to manual review, and triggering high-risk emergency plan. The large model optimization and adjustment module is used to receive manual feedback on the review results and adjust the weight parameters in the multimodal large model based on the manual feedback.

9. A terminal, characterized in that, include: The memory is used to store the decision-making process for reviewing financial documents; A processor is configured to implement the steps of the financial document review decision method as described in any one of claims 1 to 7 when executing the financial document review decision procedure.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores a financial invoice review decision program, which, when executed by a processor, implements the steps of the financial invoice review decision method as described in any one of claims 1 to 7.

Images