Large language model credibility alignment method for financial risk control

Abstract

The invention relates to the technical field of a computer system based on a specific calculation model, and discloses a financial risk control-oriented big language model credibility alignment method, which comprises the following steps of: performing bidirectional implication verification on candidate fact assertions generated by a big language model and information sources of the candidate fact assertions; after the current task passes, converting the current task into structured data and storing the structured data into a temporary fact account book established for the current task; the method comprises the following steps: constructing an internal calibration loop from information source verification to generation process constraint, verifying the consistency of candidate contents and an account book when a subsequent text is generated, and carrying out real-time intervention on a generation process of the candidate contents forming contradictions. According to the method, the internal logic self-consistency of the calculation model and the external objective facts are subjected to direct technical association, so that the brand-new session-level logic introspection and fact grounding capability of the model is improved, and the inherent generative logic forgetting defect of the model is overcome.

Description

Technical Field

[0001] This invention relates to a credibility alignment method for large language models for financial risk control, belonging to the technical field of computer systems based on specific computational models. Background Technology

[0002] Currently, when such computational models are applied to tasks requiring strict logical coherence, such as financial risk control, there is an inherent mismatch between their core operating mechanism and the task objectives. Specifically, when generating each subsequent word, the model optimizes to make the generated sequence statistically as close as possible to the distribution pattern of the training corpus, rather than strictly following the statements it has already generated in the current task. When performing tasks requiring long and coherent analysis, this operating mechanism can lead to contradictions between statements in the output text. To address this, external validation or post-processing is usually used, such as comparing the independent assertions output by the model with an external knowledge base, or conducting a full manual or automated review of the text after the model has completed all text generation.

[0003] Even when existing technologies focus on optimizing the model's classification or understanding capabilities, they often overlook the necessity of real-time constraints on model behavior during specific generation tasks, leading to persistent logical inconsistencies. For example, Chinese invention patent CN120429444B discloses a text knowledge base classification management method based on a large language model. It attempts to use the degree of interference between categories and the model's focus on discriminative information as criteria to adjust the learning rate during training, aiming to improve the model's classification accuracy for specific domain texts. However, this method essentially remains focused on optimizing the model during the training phase, aiming to improve its macroscopic accuracy in discrete classification tasks. However, it does not address the inherent limitation of large language models when continuously generating long texts. This is because their core mechanism focuses more on the statistical fluency of local texts rather than the logical consistency of global information, which may lead to inconsistencies between the preceding and following texts. Specifically, while this training-level optimization can help the model better distinguish between different categories of static features, it cannot establish a logical benchmark that the model must adhere to in the dynamic process of a single generation task. Therefore, a model optimized by such methods may still produce contradictory statements in the preceding and following text when generating financial risk control reports due to the lack of real-time internal consistency verification and constraint mechanisms, and cannot guarantee the logical coherence and factual reliability of the generated content at the conversation level.

[0004] Analysis of the aforementioned existing technologies reveals the following shortcomings: 1. Comparison with external knowledge bases is primarily used to verify the accuracy of individual factual assertions, but it cannot assess the coherence between logical nodes along the internal reasoning path of the model within a complete generation task; 2. Post-review methods allow the model to expend computational resources to generate text that will ultimately be discarded due to internal logical contradictions, resulting in inefficient allocation of computational resources. These shortcomings indicate that existing technologies lack a technical solution capable of establishing and executing dynamic, task-specific internal logical consistency constraints within the text generation process of the computational model. Therefore, the technical problem this invention aims to solve is how to design a method that enables the large language model to identify its own output key factual assertions in real time during a single text generation task, using these assertions as dynamic logical benchmarks for the current task, and performing consistency checks based on these benchmarks when generating subsequent text, thereby preventing the model from outputting content with contradictory internal logic. Summary of the Invention

[0005] This invention provides a credibility alignment method for large language models for financial risk control. Its main purpose is to solve the problem that existing technologies lack a mechanism to ensure real-time internal consistency of computational models by referring to their own output content during the generation process.

[0006] To achieve the above objectives, this invention provides a method for aligning the credibility of large language models for financial risk control, comprising the following steps: When using a large language model to perform a text generation task based on the input information source, an arbitration step is first performed on the candidate fact assertions generated by the large language model. The arbitration step includes: extracting the original text content corresponding to the candidate fact assertion from the input information source, and verifying whether there is a two-way logical implication relationship between the original text content and the candidate fact assertion. Under the condition that the bidirectional logical implication relationship holds, the candidate fact assertion is transformed into a machine-readable structured triple containing a subject, attribute, and value, and stored in a temporary fact ledger created only for this text generation task; During the process of generating subsequent text using a large language model, a natural language inference model is invoked for one or more candidate generated content to determine whether the candidate generated content constitutes a logical contradiction with any factual assertions already stored in the temporary fact ledger. In one or more steps of a decoding algorithm, if the result of the judgment is that a logical contradiction exists, the logarithmic probability value of the candidate generated content is reduced to below a preset threshold that is sufficient to prevent it from being selected by the decoding algorithm.

[0007] Preferably, the method further includes: when storing the structured triple in the temporary fact ledger, also associating and storing a source data identifier, which points to a specific data point in the input source that generated the structured triple, with the structured triple; and monitoring in parallel whether the data point pointed to by the source data identifier has changed; if a change has occurred, in the temporary fact ledger, marking the structured triple directly associated with the source data identifier, as well as all downstream fact assertions derived from the structured triple, as invalid, and preventing the structured triple marked as invalid and the downstream fact assertions from participating in subsequent logical contradiction judgments.

[0008] Preferably, the method further includes: prior to the arbitration step, determining a confidence score between 0 and 1 for the candidate factual assertion based on its wording; and reducing the logarithmic probability value of the candidate generated content to below a preset threshold, the magnitude of which is proportional to the confidence score of the factual assertion in the provisional fact ledger refuted by the candidate generated content, the relationship being determined by the following formula: ,in, This is the final log-odds value after reduction. This is the original logarithmic odds value. As a preset base penalty coefficient, and The confidence score of refuted factual assertions in the provisional factual ledger.

[0009] Preferably, it further includes a contradiction detection step, which is performed before the step of determining whether the candidate generated content constitutes a logical contradiction, and includes: pausing generation when the large language model generates text word by word to a semantic critical point of a syntactic structure, where the subject and predicate have been generated but the key object or adverbial has not yet been generated; using a prediction model, generating one or more hypothetical concept completions of the subsequent core semantics of the text segment based on the generated text segment; pre-validating the hypothetical concept completions with a temporary fact ledger to determine whether the current generation path has the risk of leading to a logical contradiction; and reducing the logarithmic probability value of the candidate generated content, which is performed in advance based on the risk determination result.

[0010] Preferably, the logarithmic probability value of the candidate generated content is reduced to below a preset threshold. Specifically, in a beam search decoding algorithm, the logarithmic probability value of the candidate generated content that constitutes a logical contradiction is directly set to a negative infinity value.

[0011] Preferably, verifying whether there is a two-way logical implication relationship between the original text content and the candidate factual assertion specifically includes: performing a first-level verification, wherein a natural language inference model is invoked to determine whether the relationship between the two is implication when the original text content is taken as a premise and the candidate factual assertion is taken as an assumption; and performing a second-level verification, wherein a natural language inference model is invoked to determine whether the relationship between the two is implication when the candidate factual assertion is taken as a premise and the core information of the original text content is taken as an assumption.

[0012] Preferably, the main body of the structured triple is an entity, the attribute is the financial characteristics of the entity, and the value is a specific statement of the financial characteristics.

[0013] Preferably, a confidence score is determined for a candidate factual assertion by a confidence classifier that generates the confidence score based on the analysis of modal verbs and adverbs contained in the candidate factual assertion.

[0014] Preferably, the step of generating candidate factual assertions is implemented by using a lightweight text classification model combined with keyword matching and regular expressions. The step is configured to capture statements in the text that contain explicit financial indicators and values, strong causal correlation words, or qualitative judgments.

[0015] Preferably, after marking the downstream fact assertion as invalid, the method further includes guiding the large language model to regenerate a completely new text sequence based on the changed data points.

[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. When using a large language model to process text, the first step is to verify the bidirectional logical implication relationship between the original information source and the assertions initially generated by the model. Only then are the verified assertions stored in a temporary fact ledger established for this task. In the subsequent text generation process, any candidate content must be consistent with the assertions in the ledger that have been verified by the information source. This process directly links the pre-arbitration stage of fact determination with the consistency constraint stage of content generation, forming a logically closed and traceable internal calibration loop from the information source to the final output. As a result, the text generation behavior of the model in a single task changes from an unconstrained sequential construction along a probability path to a process of continuous self-correction by referring to verified facts. This establishes a direct technical connection between the evolution of the internal state of the computational model and the external objective information source when performing the task.

[0017] 2. When storing factual assertions in the fact ledger, not only is their content recorded, but a confidence level is also assigned to them, and a source data identifier pointing to their original source is associated with them. When the model generates subsequent content and performs consistency checks, the intensity of the intervention is directly related to the confidence level of the refuted fact. At the same time, a parallel monitoring mechanism monitors the data status pointed to by the source data identifier. Once the source data changes, all downstream factual assertions in the ledger derived from it will immediately become invalid. This method of synchronously managing the confidence and timeliness attributes of assertions enables the computational model to overcome complex inputs that mix deterministic and non-deterministic static and dynamic information. Its internal logical constraints are no longer singular and rigid, but rather it gains a multi-dimensional adaptability that matches the complexity of the external information environment.

[0018] 3. When the text generation reaches the preset semantic threshold in the sentence, the generation will be paused, and a prediction model will be used to generate several hypothetical concepts to complete the subsequent content. These hypothetical concepts will be quickly pre-verified against the fact ledger to determine whether there is a risk of logical contradiction in the current generation path. Based on the risk judgment result, the generation process of the path will be intervened in advance. This move decomposes the consistency verification action that was originally located at the end of the generation into a risk detection stage in the middle of the generation and a final verification stage. Therefore, the consistency verification mechanism not only ensures the logical accuracy of the output content, but also acts as a control signal for optimizing the allocation of computing resources. It acts on the generation process itself to guide the computing model to actively avoid those generation paths that will invest computing resources but are likely to be discarded due to logical contradictions. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process and internal calibration loop of the method of the present invention; Figure 2 This is a graph showing the performance impact and calibration curves of the basic penalty coefficient in this invention. Figure 3 This is a sequence diagram of the component interactions for the fact assertion arbitration step of this invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the scope of protection of the present invention.

[0021] This invention provides a credibility alignment method for a large language model for financial risk control. This software method runs on a computer system based on a specific computational model. It mainly includes a fact assertion arbitration step, a dynamic fact storage step, a consistency verification step, and a generation process intervention step. These steps work together to construct a real-time calibration loop within the computational model, from source verification to generation process constraints. Its purpose is to address the problem that the computational model's inherent sequence generation mechanism may lead to logical contradictions in the output content when performing long text generation tasks requiring high logical coherence, such as financial risk analysis. In a specific application scenario, this method is used to generate an analysis report on the short-term solvency of a specified company based on its quarterly financial report as input. To handle the risk of misidentifying speculative statements in the input source as facts or distorting details of the original data during the summarization process, this method first performs a fact assertion arbitration step. After the large language model processes the input source, it generates a candidate fact assertion, such as the company's... When the debt-to-equity ratio was 60% in the third quarter, this assertion was not directly adopted but placed in a temporary pending area, accompanied by a source identifier pointing to its specific location in the original financial statement document. Subsequently, a source-assertion two-way implication verification mechanism, which serves as the core of arbitration, was invoked. It performs double verification through a natural language inference model fine-tuned from financial corpus. The first verification is based on the original text extracted from the source, Section 5.2 of the financial statement, which shows that as of the end of the reporting period, the company's total liabilities were 600 million yuan and shareholders' equity was 400 million yuan, as a premise, and candidate facts. The first step is to assume that the company's debt-to-equity ratio in the third quarter is 60%. The second step is to determine whether the former logically implies the latter, in order to confirm that the assertion has a factual basis. The third step is to perform the reverse verification, taking the candidate factual assertion as a premise and the core information of the original source as an assumption, to determine whether the former logically implies the latter, in order to confirm that the assertion has not omitted or distorted key information in the original text. Only when the candidate factual assertion passes both of these layers of implication verification, proving that it constitutes an informational equivalent paraphrase relationship with the source, is the arbitration step determined to be a qualified factual assertion.

[0022] After a candidate fact assertion passes arbitration, the dynamic fact storage step is executed. The system normalizes this qualified assertion, converting it into a machine-readable structured triple consisting of a subject, attribute, and value, specifically {subject: the company, property: debt-to-equity ratio, value: 60%}. This triple is then stored in a temporary fact ledger dynamically created in memory solely for this analysis report generation task. To address the potential for dynamic changes in information within financial scenarios, when storing this structured triple in the ledger, the system also assigns a source data identifier pointing to its originating data point, such as a financial identifier. Report Q3_Item_5.2, associated with this triplet for storage. A parallel source data monitor will track whether the data point pointed to by this identifier has changed. If the value of this item changes due to subsequent input of updated financial data, the monitor will trigger a logical failure propagation mechanism. This not only marks the triplet directly derived from this data in the ledger as invalid, but also traverses the ledger to mark all downstream fact assertions derived from this invalid fact as invalid as well, to ensure that the consistency basis of the model can remain synchronized with changes in external facts. As the analysis report generation process continues, when the model needs to generate subsequent text, consistency... The consistency verification step and the intervention step in the generation process begin. In an implementation using a beam search decoding algorithm, when the model generates a candidate content, such as "Therefore, the company has low financial leverage and short-term debt repayment risk is controllable," this candidate content is not immediately output. Instead, it first passes through a consistency verifier. The verifier calls a natural language inference model to compare the candidate content with every fact assertion already stored in the temporary fact ledger to determine if there is a logical contradiction. For example, if the ledger already contains the record {subject: the company, property: short-term loans, value: significant increase}... The natural language inference model will determine whether the conclusion that the short-term debt repayment risk is controllable in the candidate content constitutes a logical contradiction with the fact. Once the judgment result is that a logical contradiction exists, the generation process intervention step is triggered. The system will intervene in the generation process of the candidate content in real time. Specifically, it will impose a penalty on the logarithmic probability value in the beam search algorithm, for example, by setting it directly to a negative infinity value, or reducing it to a preset threshold that is sufficient to prevent it from being selected by the decoding algorithm. This operation prunes the candidate content with logical contradiction on the generation path, thereby preventing the content that constitutes a logical contradiction from being finally output.

[0023] Furthermore, to address the ambiguity and uncertainty commonly found in financial texts, this method can also integrate a confidence-weighted consistency check method. Before a factual assertion is entered into the ledger, a confidence classifier analyzes its statement. For the assertion that the company's cash flow appears to be weakening, the classifier assigns a confidence score between 0 and 1, such as 0.4, based on modal adverbs like "appears." Accordingly, in subsequent consistency checks and when applying contradiction penalties, the strength of the penalty will be correlated with the confidence score of the refuted factual assertion, and the relationship can be determined by the following formula: ,in, The final logarithmic probability value after penalty. This is the original logarithmic odds value. As a preset base penalty coefficient, and The confidence score is assigned to the refuted factual assertions in the provisional fact ledger. Refuting a definitive fact with a confidence score of 1.0 incurs a stronger penalty, while refuting a speculative assertion with a confidence score of 0.4 incurs a weaker penalty. This allows the model to reason more flexibly about uncertain information without violating core facts. Furthermore, to improve computational efficiency, this method introduces a contradiction detection mechanism. This mechanism executes before full consistency verification. It pauses generation when the model generates text word by word to a semantic critical point in the syntactic structure, for example, after generating a statement based on the company's strained cash flow. Following this segment, a lightweight predictive model will quickly generate several hypothetical concepts to complete the core semantics of future credit rating, such as {upgrade, remain unchanged, downgrade}. The system will then use these hypothetical concepts to perform a rapid pre-verification against the factual ledger. If it finds that all high-probability subsequent concepts potentially contradict the fact of tight cash flow in the ledger, it will determine that the current generation path has the risk of leading to a logical contradiction. Based on this risk judgment, a punitive adjustment will be applied to the score of the generation path in advance, thereby guiding the decoding algorithm to avoid this generation path that is likely to lead to a logical contradiction before investing more computing resources.

[0024] Example 1: In an automated due diligence scenario for a potential M&A transaction, a computational model is configured to generate a comprehensive risk assessment report based on a dynamic data room containing hundreds of documents. The data room is unique in that its initial components consist of financial forecasts and business outlooks provided by the target company's management, while later components include legal documents from a third-party auditing firm containing numerous risk warnings and disclosures of pending litigation. This situation challenges the computational model's ability to maintain logical consistency in the final report when handling conflicts in the chronological order and content of information flow. At the initial stage of the task, the model first processes the early financial forecast documents. When generating the summary text, the method's fact-assertion arbitration step is triggered, and a candidate fact... The assertion, namely that the target company's projected compound annual growth rate (CAGR) of 15% over the next three years, is sent to the review area after generation. Through a source-assertion bidirectional implication verification mechanism, it is confirmed that the assertion is supported by data in the original source file and does not distort the original meaning. Subsequently, the assertion is transformed into a structured triple {subject: target company, property: projected CAGR, value: 15%} by the dynamic fact storage step and stored in a temporary fact ledger created for this report generation task. Based on this, the model generates a positive description of the target company's growth potential in the opening section of the report. All these early-generated text contents successfully pass the consistency verification step because they do not logically contradict this fact assertion in the ledger.

[0025] As the processing progresses, the model begins analyzing a later-added legal due diligence report. During this process, the fact-assertion arbitration step identifies another key piece of information. After two-way implication verification, a new fact-assertion is generated: the target company faces a potential environmental lawsuit with a maximum possible payout of 80% of the company's net assets. This is formalized as {subject: target company, property: potential environmental litigation liability, value: 80% of net assets} and stored in the provisional fact ledger. At this point, a common technical problem arises: in pursuing local textual fluency, the computational model tends to perpetuate a previous optimistic tone. A model lacking the mechanism of this invention might generate an error in the report's conclusion section. As summarized above, the target company has a sound financial foundation, and the overall risk of this acquisition is controllable. However, under this method, this candidate content is immediately processed by a consistency check step after generation. The natural language inference model compares this candidate content with all records in the provisional fact ledger and determines that the candidate content logically contradicts the factual assertions in the ledger regarding potential environmental litigation liabilities. Subsequently, an intervention step in the generation process is activated. In a beam search decoding algorithm, the system sets the logarithmic probability value of this logically contradictory candidate content to a negative infinity, causing this generation path to be abandoned immediately. This intervention forces the computational model to abandon the path that continues the previous tone and instead explore other paths related to the provisional fact ledger. The model generates a logically consistent conclusion by ensuring all facts remain logically consistent. Ultimately, it produces an alternative, logically consistent conclusive text. For example, while the target company demonstrates business growth potential, disclosed potential environmental litigation constitutes a significant financial risk to the transaction and could have a substantial negative impact on the post-merger company value. Thus, through the synergistic effect of pre-assessment of factual assertions and real-time consistency constraints during the generation process, the final analysis report not only avoids logical contradictions caused by the order of information input but also transforms an open-domain text continuation task into a constrained reasoning process within a closed logical boundary. Its output text sequence therefore possesses the inherent qualities required in the field of financial risk control. Logical coherence; Based on the scenario of this embodiment, at time T1 after the automated due diligence system completes the generation of the preliminary report, an audit document in the data room, which serves as a core information source, is updated. Specifically, a contingent liability of the target company is recognized as an actual liability, causing its debt-to-asset ratio to change from 60% to 75%. The source data monitor built into the system detects this change in real time by comparing the checksum of the information source document and immediately triggers the logical failure propagation mechanism. The factual assertion {subject: target company, property: debt-to-asset ratio, value: 60%) from the old version of the document in the temporary fact ledger, as well as all downstream factual assertions derived from this assertion, are marked as invalid.Subsequently, the large language model was guided to automatically generate a completely new text sequence based on the changed data points, consistent with the current factual basis. This replaced all analyses and conclusions in the report based on outdated data, thus ensuring the timeliness and accuracy of the final report.

[0026] Example 2: To objectively verify the effectiveness of the method of the present invention in suppressing self-contradictions in the content generated by the computational model, this example designed and executed a comparative experiment. The purpose was to quantitatively evaluate the improvement in the logical consistency of the output text when the computational model performs a long text generation task containing inherent information conflicts after the introduction of the method. This experiment established a standardized evaluation environment. The experimental platform was based on the same pre-trained large language model with 7 billion parameters, and two different running modes were configured as experimental groups. The control group used the model's native configuration combined with a standard beam search decoding algorithm with a beamwidth set to 5. Text generation was performed; the experimental group, based on the control group, integrated the complete method of this invention, which includes a fact assertion arbitration step, a dynamic fact storage step, a consistency verification step, and a generation process intervention step; the dataset used in the experiment was an internally constructed financial analysis corpus containing 100 test cases. Each test case contained an input source of about 1,000 words. Its structure was set so that the first half of the document contained 5 explicit positive fact assertions, while the second half introduced 5 negative fact assertions that constituted potential logical contradictions with the former. The experimental task required the model to generate a summary analysis report of no less than 500 words for each test case.

[0027] The evaluation metric for the experiment was the logical contradiction rate, defined as the percentage of reports containing at least one self-contradictory statement in the final generated summary analysis report out of all test cases. An independent automated judgment program based on a natural language inference model was used for initial screening, and cross-validation was performed by three annotators. During the experiment, for a typical test case, the input source clearly stated in the first half that Company A had sufficient cash flow, but disclosed in the second half that Company A had a high short-term debt repayment risk. When generating the report, the control group model generated a statement at the beginning of the report that was consistent with the information in the first half, indicating that Company A's cash flow was healthy, but in the conclusion of the report, it generated a sentence that contradicted the preceding statement, concluding that the company's short-term debt pressure was low. In contrast, the experimental group... When processing the same use case, the model's temporary fact ledger already contains two arbitrated factual assertions: {subject: Company A, property: cash flow, value: sufficient} and {subject: Company A, property: short-term debt repayment risk, value: relatively high}. Therefore, when the decoding algorithm generates the candidate content "the company's short-term debt pressure is low" in the conclusion section, the consistency check step determines that this content constitutes a logical contradiction with the factual assertion "the short-term debt repayment risk is relatively high" in the ledger, and immediately triggers the generation process intervention step, which applies a penalty to the logarithmic probability value of the candidate content, causing it to be abandoned by the decoding algorithm. The model finally generates alternative text that does not contradict any of the facts in the ledger. See Table 1, which lists the processing of 10 representative test cases.

[0028] Table 1: Comparison of the behavior of different test groups when handling representative test cases.

[0029] After processing all 100 test cases, statistics showed that 87 out of the 100 reports generated by the control group were found to contain at least one logical contradiction, a logical contradiction rate of 87%. In contrast, only 4 out of the 100 reports generated by the experimental group using the method of this invention contained logical contradictions due to misjudgment by the natural language inference model, a logical contradiction rate of 4%. The experimental data demonstrates that the logical consistency of the output text from the experimental group using the method of this invention is significantly improved compared to the control group using the standard generation method, proving that this method can effectively improve the text generation process of the computational model. By establishing dynamic internal logical consistency constraints, contradictory text content is suppressed. To further verify the effectiveness of the method in terms of computational resource utilization, a supplementary test is added to the experiment in Example 2. This test divides the experimental group into two subgroups: subgroup A enables the contradiction detection step, while subgroup B disables it, with the remaining configurations identical to the experimental group in Example 2. The test metric is the total computational resource consumption required to process all 100 test cases, measured in floating-point operations (FLOPs). The test results show that the total consumption of subgroup B is... FLOPs, while the total consumption of experimental subgroup A was FLOPs, reducing computational resource consumption by approximately 33.3%; the data shows that the contradiction detection step reduces the computational resources invested by the computational model in generating invalid text sequences that will eventually be discarded by identifying and pruning candidate content that is likely to lead to logical contradictions in the early stages of the generation path.

[0030] Example 3: This example combines Figures 1 to 3 This paper explains a credibility alignment method for a large language model oriented towards financial risk control, such as... Figure 1 As shown, the process begins by receiving input sources, such as corporate financial reports. The Large Language Model (LLM) then initially extracts factual statements to generate candidate factual assertions. These candidate assertions are not directly adopted but instead undergo a bidirectional implication verification process between the source and the assertion. If verification fails, the process retryes. If verification succeeds, the assertion is transformed into a structured triple and stored in a temporary fact ledger, which serves as the dynamic logical baseline for this task. In parallel, a source data monitoring and failure propagation mechanism detects changes in the source and invalidates associated facts in the ledger. During subsequent text generation, the system performs a contradiction detection before the LLM generates subsequent candidate content to predict and avoid potentially contradictory generation paths. The generated candidate content must undergo a consistency verification process with the temporary fact ledger. If there is no contradiction, the content is accepted and an analysis report is ultimately output. If a contradiction exists, an intervention process is triggered, adjusting the intervention intensity based on the confidence-weighted verification results to reduce the probability of generating contradictory content and guide the LLM to regenerate text consistent with the ledger logic.

[0031] like Figure 2 As shown, this provides a calibration basis for a key parameter in the intervention stage of the generation process. The horizontal axis in the figure represents the base penalty coefficient increasing from 1.0 to 20.0. The left side of the main vertical axis represents the conflict avoidance rate (%), and the right side of the secondary vertical axis represents the normalized text fluency. The solid curve in the graph shows that the conflict avoidance rate increases with the base penalty coefficient. The coefficient increases monotonically with increasing base penalty coefficient, while the dashed curve shows that text fluency increases with increasing base penalty coefficient. The curves increase and then decrease monotonically. The trends of the two curves clearly reveal the technical trade-off between improving logical consistency and maintaining the natural fluency of the text, providing a quantitative reference for selecting an engineering balance between the two.

[0032] like Figure 3 As shown in the figure, this diagram illustrates the collaborative workflow among multiple system components, including a large language model, input source, original text extractor, natural language inference model, confidence classifier, and review area. The process begins with the large language model submitting a candidate fact assertion to the review area and requesting the corresponding original text content from the original text extractor. After obtaining the original text, the large language model sends the original text and assertion together to the natural language inference model to perform the first and second layers of verification. If the bidirectional entailment verification passes, the assertion is sent to the confidence classifier to calculate the confidence score, and the pass result with the confidence score is sent to the review area. If the verification fails, the natural language inference model directly returns a failure result to the review area. Finally, based on the received results, the review area returns the final status of whether the assertion is accepted or rejected to the large language model.

[0033] Example 4: In the preliminary stage of deploying the method of the present invention in an automated credit risk monitoring system that processes real-time financial news streams, its core natural language inference model and confidence classifier need to possess lightweight and domain-specific attributes. Furthermore, the basic penalty coefficient used in the method to adjust the intensity of contradiction penalties... The specific numerical setting affects the balance between maintaining logical consistency and ensuring the naturalness of generated text, requiring a reproducible procedure to determine. To meet this requirement, this embodiment adopts a systematic offline calibration and model building procedure. First, for the construction of the natural language inference model, the procedure adopts the knowledge distillation technique, selecting a large-scale general pre-trained language model as the teacher model and constructing a dedicated dataset containing 100,000 pairs of sentences in the financial field. These pairs are labeled as implied, neutral, or contradictory relationships. Subsequently, a lightweight MobileBERT is used as the basic architecture of the student model. Using the aforementioned dedicated dataset, the ability of the teacher model to judge the logical relationships of financial text is distilled into the student model, thereby obtaining a natural language inference model with fast inference speed, low resource consumption, and optimized for financial context. Similarly, the construction of the confidence classifier is also completed by supervised fine-tuning on the same lightweight model architecture using a dataset containing 50,000 financial news sentences labeled with confidence scores.

[0034] Secondly, regarding the base penalty coefficient The calibration procedure involves performing a gradient optimization experiment. The experiment uses 1000 pairs of sentences labeled as contradictory from the aforementioned dataset specifically designed for natural language inference models as the test set. The experimental procedure is as follows: the first sentence of each sentence pair is used as the model's input, and the model is instructed to generate subsequent text. Simultaneously, the base penalty coefficient is... The value starts at 1.0, increases in increments of 1.0, up to 20.0, and in each... Under the given values, all 1000 test cases were run completely, and two performance metrics were recorded: one was the contradiction avoidance rate, which is the percentage of test cases where the model successfully prevented the generation of the second contradictory sentence; the other was text fluency, which is the average perplexity score calculated for all successfully generated non-contradictory text outputs. Experimental data shows that when As the value increased from 1.0 to 15.0, the conflict avoidance rate monotonically increased from 65.3% to 99.5%, while the average perplexity score of text fluency only slightly deteriorated. When the value increases from 15.0 to 20.0, the increase in contradiction avoidance rate is less than 0.2%, while the average perplexity score deteriorates. Based on this data, in order to achieve a technical balance between contradiction avoidance rate and text fluency, this embodiment will adjust the base penalty coefficient. The engineering value is determined to be 15.0. By executing the above procedures, this embodiment not only obtains a lightweight and reliable core model component suitable for the target application scenario, but also transforms a key parameter affecting system behavior from a value set based on experience into a specific value determined through standardized experimental procedures and quantitative data analysis, thereby providing technical details for the reproducibility and engineering deployment of the entire method.

[0035] Example 5: When the method of the present invention is applied to financial intelligence analysis scenarios involving a large amount of unstructured text, such as analyst comments and market rumors, in order to ensure that the factual assertions entering the temporary fact ledger have high credibility, this example also includes an acceptance threshold calibration procedure for the two-way implication verification in the factual assertion arbitration step; the procedure is executed on an offline verification dataset containing 10,000 pairs of financial domain text sentence pairs, where each sentence pair consists of a source text and a candidate factual assertion, and has been labeled with a factual fidelity score between 0 and 100 by domain analysts.

[0036] During the execution of this calibration procedure, the system inputs each sentence pair in the verification dataset into the constructed natural language inference model to perform bidirectional implication verification and records the confidence scores of the implication relations output by the model in both verification directions. Subsequently, the system uses the geometric mean of these two confidence scores as the final comprehensive implication score and plots a performance curve between this comprehensive implication score and the manually labeled fact fidelity score. By analyzing this performance curve, the system selects the position where a preset balance point between precision and recall is reached, and sets the comprehensive implication score corresponding to this point, i.e., 0.95, as the final acceptance threshold for the fact assertion arbitration step. In subsequent online operations, only when the comprehensive implication score of a candidate fact assertion and its source is greater than or equal to the calibration threshold of 0.95 is it allowed to be stored in the temporary fact ledger. This procedure provides a data-driven judgment benchmark for the admission control of fact assertion.

[0037] Example 6: When applying the method of the present invention to a financial intelligence analysis scenario involving a large amount of unstructured text, such as analyst comments and market rumors, to ensure that the factual assertions entering the temporary fact ledger have high confidence, this example also includes an acceptance threshold calibration procedure for the two-way implication verification in the factual assertion arbitration step; this procedure is executed on a dedicated offline verification dataset containing 10,000 pairs of text sentences in the financial domain, where each sentence pair consists of a source text and a candidate factual assertion, and has been labeled with a factual fidelity score between 0 and 100 by domain analysts.

[0038] During the execution of this calibration procedure, the system inputs each sentence pair in the verification dataset into the constructed natural language inference model to perform bidirectional implication verification and records the confidence scores of the implication relations output by the model in both verification directions. Subsequently, the system uses the geometric mean of these two confidence scores as the final comprehensive implication score and plots a performance curve between this comprehensive implication score and the manually labeled fact fidelity score. By analyzing this performance curve, the system selects the position where a preset balance point between precision and recall is reached, and sets the comprehensive implication score corresponding to this point, i.e., 0.95, as the final acceptance threshold for the fact assertion arbitration step. In subsequent online operations, only when the comprehensive implication score of a candidate fact assertion and its source is greater than or equal to the calibration threshold of 0.95 is it allowed to be stored in the temporary fact ledger. This procedure provides a quantifiable, data-driven judgment benchmark for the admission control of fact determination.

[0039] To further verify the necessity and technical effectiveness of the internal calibration loop used to ensure the logical consistency of the generated content as a core technical feature in the method of the present invention, this application also provides the following comparative examples.

[0040] Comparative Example 1: This comparative example aims to verify the technical performance of existing large language models in handling financial analysis tasks containing inherent information conflicts when the internal calibration loop from source verification to generation process constraints of this invention is lacking. The experimental conditions used in this comparative example are consistent with the experimental group in Example 2, except for the differences explicitly stated below. Specifically, this comparative example uses the same pre-trained large language model with 7 billion parameters as in Example 2, and is configured with a standard beam search decoding algorithm with a beamwidth of 5. The essential difference is that the large language model in this comparative example does not integrate any of the large language model credibility alignment methods for financial risk control claimed in this invention, which include fact assertion arbitration, dynamic fact storage, consistency verification, and generation process intervention. Instead, it adopts a conventional, unconstrained text generation method in the art. The task of this comparative example is to require the model to generate text based on a document constructed for this experiment. Based on the comprehensive due diligence report on Target Company C (input source), generate an investment risk summary of no less than 500 words. This input source is specially designed, and the technical facts it contains may have potential logical conflicts in different sections, as follows: Source Fact 1 (from the Marketing and Sales Department report): It clearly states that Target Company C's main product, the X-series chip, has successfully signed a five-year exclusive supply agreement with Jushi Automotive, a leading downstream automaker. It is expected to account for more than 80% of Jushi Automotive's procurement share of similar parts, providing a stable guarantee for the company's revenue for the next five years; Source Fact 2 (from the Supply Chain Risk Review Department report): It clearly states that the upstream company S, the sole supplier of K material, the core raw material for the X-series chip, has recently been ordered to suspend production for rectification due to environmental issues, and the date of resumption of production is still uncertain. Target Company C's K material inventory can only maintain normal production for 30 days, and no alternative supplier with the same quality and capacity has yet been found.

[0041] During the experiment, the large language model using the conventional technical path of this comparative example received the above input sources and began generating an investment risk summary. In the opening section of the report, the model first generated the following positive statement based on source fact 1: The core competitiveness of target company C lies in the stable market position of its main product, the X-series chips. The company has secured a five-year exclusive supply agreement with industry giant Jushi Automotive, which greatly guarantees the certainty of the company's future revenue and its potential for continued growth. Subsequently, when the model's processing progressed to the latter part of the input sources, it encountered source fact 2 regarding supply chain disruptions. In the conclusion section of the report, the model generated the following candidate content: In summary, target company C, with its long-term exclusive supply agreement for its key products, demonstrates strong market control and financial stability, and its investment risk level is assessed as low. Due to the lack of the real-time internal consistency constraint mechanism of this invention, although the above candidate content is consistent with the positive statement at the beginning of the report in a local context, it constitutes a serious logical contradiction with source fact 2 regarding the disruption of the supply of core raw materials, which the model has already processed. The candidate content was selected and finally output by the beam search decoding algorithm. After the complete output report was submitted to a downstream automated portfolio management system, the system failed to identify the deep logical contradictions in the text and mistakenly maintained the risk exposure of target company C at a low level. This significant risk omission was not discovered until it was manually reviewed 15 business days later, triggering an emergency position adjustment, as shown in Table 2.

[0042] Table 2: Comparison of Experimental Results.

[0043] The experimental results of this comparative example show that, in the absence of the internal calibration loop proposed in this invention, which extends from source verification to generation process constraints, even with the same basic model, conventional text generation methods, in their technical principles, are indeed unable to handle inputs containing conflicting information in a single generation task, and may produce output content containing inherent logical contradictions. As a result, they cannot meet the requirements of high logical coherence for serious application scenarios such as financial risk control.

[0044] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A credibility alignment method for large language models oriented towards financial risk control, characterized in that, Includes the following steps: When using a large language model to perform a text generation task based on the input information source, an arbitration step is first performed on the candidate fact assertions generated by the large language model. The arbitration step includes: extracting the original text content corresponding to the candidate fact assertion from the input information source, and verifying whether there is a two-way logical implication relationship between the original text content and the candidate fact assertion. Under the condition that the bidirectional logical implication relationship holds, the candidate fact assertion is transformed into a machine-readable structured triple containing a subject, attribute, and value, and stored in a temporary fact ledger created only for this text generation task; During the process of generating subsequent text using a large language model, a natural language inference model is invoked for one or more candidate generated content to determine whether the candidate generated content constitutes a logical contradiction with any candidate fact assertions already stored in the temporary fact ledger. In one or more steps of a decoding algorithm, if the result of the judgment is that a logical contradiction exists, the logarithmic probability value of the candidate generated content is reduced to below a preset threshold that prevents it from being selected by the decoding algorithm.

2. The credibility alignment method for a large language model oriented towards financial risk control according to claim 1, characterized in that, The method also includes: when storing the structured triple in the temporary fact ledger, associating and storing a source data identifier, which points to a specific data point in the input source that generated the structured triple, with the structured triple; and monitoring in parallel whether the data point pointed to by the source data identifier has changed; if a change has occurred, in the temporary fact ledger, marking the structured triple directly associated with the source data identifier, as well as all downstream fact assertions derived from the structured triple, as invalid, and preventing the structured triple marked as invalid and the downstream fact assertions from participating in subsequent logical contradiction judgments.

3. The credibility alignment method for a large language model oriented towards financial risk control according to claim 1, characterized in that, Also includes: Prior to the arbitration step, a confidence score between 0 and 1 is assigned to each candidate factual assertion based on its wording; and the logarithmic probability value of the candidate generated content is reduced to below a preset threshold, the magnitude of which is proportional to the confidence score of the factual assertion in the provisional fact ledger refuted by the candidate generated content, the relationship being determined by the following formula: ,in, This is the final log-odds value after reduction. This is the original logarithmic odds value. As a preset base penalty coefficient, and The confidence score of refuted factual assertions in the provisional factual ledger.

4. The credibility alignment method for a large language model oriented towards financial risk control according to claim 1, characterized in that, It also includes a contradiction detection step, which is performed before the step of determining whether the candidate generated content constitutes a logical contradiction. This step includes: pausing generation when the large language model generates text word by word to a semantic critical point of a syntactic structure, where the subject and predicate have been generated but the key object or adverbial has not yet been generated; using a prediction model, generating one or more hypothetical concept completions for the subsequent core semantics of the generated text fragment based on the generated text fragment; pre-validating the hypothetical concept completions with a temporary fact ledger to determine whether the current generation path has the risk of leading to a logical contradiction; and reducing the log-probability value of the candidate generated content, which is performed in advance based on the risk determination result.

5. The credibility alignment method for a large language model oriented towards financial risk control according to claim 1, characterized in that, The logarithmic probability value of candidate generated content is reduced to below a preset threshold. Specifically, in a beam search decoding algorithm, the logarithmic probability value of candidate generated content that constitutes a logical contradiction is directly set to a negative infinity value.

6. The credibility alignment method for a large language model oriented towards financial risk control according to claim 1, characterized in that, Verifying whether there is a two-way logical implication relationship between the original text content and the candidate factual assertions specifically includes: performing a first-level verification, which involves calling a natural language inference model to determine whether the relationship between the two is implication when the original text content is taken as a premise and the candidate factual assertion is taken as an assumption; and performing a second-level verification, which involves calling a natural language inference model to determine whether the relationship between the two is implication when the candidate factual assertion is taken as a premise and the core information of the original text content is taken as an assumption.

7. The credibility alignment method for a large language model oriented towards financial risk control according to claim 1, characterized in that, The main body of a structured triple is an entity, the attributes are the financial characteristics of that entity, and the values ​​are specific statements of those financial characteristics.

8. The credibility alignment method for a large language model oriented towards financial risk control according to claim 3, characterized in that, Determining a confidence score for a candidate factual assertion is accomplished using a confidence classifier, which generates the confidence score based on the analysis of modal verbs and adverbs contained in the candidate factual assertion.

9. The credibility alignment method for a large language model oriented towards financial risk control according to claim 1, characterized in that, The step of generating candidate factual assertions is achieved through a lightweight text classification model that combines keyword matching and regular expressions. The step is configured to capture statements in the text that contain explicit financial indicators and values, strong causal words, or qualitative judgments.

Images