A RAG Financial Credit Decision-Making Method Based on Policy Mask Constraint Decoding

Abstract

This invention belongs to the field of financial technology and provides a RAG financial credit decision-making method based on policy mask constraint decoding. The method includes: acquiring a structured financial credit policy codebook; filtering credit parameter values ​​that can be encoded into individual tokens by a pre-trained word segmenter; constructing a set of effective policy token IDs; and then, through a mask vector matched with a large language model vocabulary, adding the original logits vector to the mask vector during decoding to force the output tokens to be constrained within the effective set, achieving compliance in a single decoding operation. Simultaneously, it adds dynamic policy updates, parameter type sub-masks, semantic offset error recovery, and a policy codebook structure verification mechanism. This invention ensures the compliance of credit decision text from the source, improves decision-making efficiency, shortens policy adjustment response time, and reduces decision interruption rates.

Description

Technical Field

[0001] This invention relates to the field of financial technology, specifically to a RAG financial credit decision-making method based on policy mask constraint decoding. Background Technology

[0002] Financial credit decisions are a core aspect of risk management for financial institutions, and their outcomes must strictly adhere to structured policy requirements such as regulatory policies and internal credit rules (e.g., credit amount tiers, term tiers, interest rate ranges). Traditional credit decisions rely on manual interpretation of policies and matching of customer information, which suffers from low efficiency, high subjectivity, and susceptibility to compliance risks arising from misunderstandings of policies.

[0003] With the development of artificial intelligence technology, Retrieval Augmentation (RAG) technology combined with Large Language Modeling (LLM) has been applied to automate credit decision-making: by retrieving relevant policy texts, the large language model generates decision suggestions. However, existing technologies have the following key drawbacks: First, the content generated by the large language model relies on probabilistic prediction, which may output parameter values ​​outside the policy (such as exceeding the prescribed amount tier), requiring additional manual verification, thus violating the original intention of automation; second, financial policies are often dynamically updated, and existing constraint decoding methods (such as rule filtering) are difficult to adapt to new policies quickly, and cannot accurately constrain the structured characteristics of credit parameters (such as fixed tiers such as 500,000-1,000,000 yuan for 12 months); third, when the generated content is close to the policy boundary or encounters ambiguity, there is a lack of effective error recovery mechanisms, which can easily lead to decision interruption or the generation of invalid text.

[0004] Large-scale modeling combined with retrieval-enhanced generation (RAG) has been widely applied in fields such as financial question answering. However, in scenarios requiring precise numerical output, such as credit granting decisions, directly using large models to generate key information like amounts and interest rates often leads to "illusion" phenomena due to their free decoding nature. This results in outputs exceeding regulatory or internal bank policies, directly causing serious compliance risks and potential asset losses. Existing industry solutions generally employ a post-processing rule-based verification + rollback model: first, the large model freely generates complete credit suggestion text; then, key numbers are extracted from the text using regular expressions; finally, an external rule engine verifies the extracted numbers. If non-compliant, the generated result is discarded, guiding the model to regenerate, or it rolls back to a preset conservative value. This approach has significant drawbacks: first, the multiple rounds of generation and verification increase decision-making time, requiring an average of 2-3 rounds to obtain a compliant result, failing to meet real-time credit granting requirements; second, forced rollback or modification (e.g., forcibly changing the model's suggested limit of 500,000 to 10,000) disrupts the text's coherence and interpretability, potentially triggering customer complaints; third, this mechanism relies on post-processing, making it vulnerable to carefully designed prompt injection attacks. This can be bypassed by injection, posing a security risk.

[0005] Currently, there is no publicly available technology that can perform hard masking constraints within the decoder based on the bank's real-time updated policy codebook, and effectively handle word segmentation problems caused by different word segmenters.

[0006] Therefore, a RAG financial credit decision-making method based on policy mask constraint decoding is needed to solve the various problems existing in the traditional methods and existing technologies mentioned above. Summary of the Invention

[0007] This invention provides a RAG financial credit decision-making method based on policy mask constraint decoding, aiming to solve the problems of low efficiency, strong subjectivity, compliance risks caused by policy misunderstandings, and the potential for large language models to output policy-related parameter values, difficulty in quickly adapting to new policies, and lack of effective error recovery mechanisms in the automated application of credit decision-making. This method obtains a structured financial credit policy codebook, initializes a set of valid policy term IDs, and uses a pre-trained word segmenter to encode and filter the strings in the codebook, adding qualified term IDs to the set of valid policy term IDs. During the large language model decoding process, the original logits vector is constrained by a mask vector, enabling the generation of compliant financial credit decision text in a single decoding operation. Furthermore, this method supports dynamic policy updates, sub-mask grouping, error recovery, and structured verification of the policy codebook to ensure the accuracy, compliance, and real-time performance of credit decisions.

[0008] This invention provides a RAG financial credit decision-making method based on policy mask constraint decoding, comprising the following steps: Step 1: Obtain a structured financial credit policy codebook, which includes string representations of multiple allowed credit parameter values; Step 2: Initialize the set of valid policy term IDs; the set of valid policy term IDs is an empty set of integers used to store term IDs that have been verified and can be used for constraint decoding; Step 3: Using a pre-trained word segmenter that matches the large language model, encode each string in the financial credit policy codebook, filter out the strings that can be encoded into a single word, and add the word ID corresponding to the string to the set of effective policy word IDs; Step 4: During the large language model decoding process, the pre-constructed mask vector is added to the original logits vector output by the large language model to obtain the logits vector after masking. This forces the output lexical units to be constrained to the set of valid policy lexical unit IDs during the generation process, thereby achieving the generation of compliant financial credit decision text in a single decoding operation.

[0009] Furthermore, the credit parameter values ​​include at least one of the following: credit amount level, credit period level, interest rate level, and necessary conjunctions used to form natural language sentences.

[0010] Furthermore, the screening step in step three further includes: Perform encoding operations on the string to generate a sequence of lexical IDs consisting of one or more lexical IDs; Determine if the length of the generated word ID sequence is equal to 1; If the length of the lexical ID sequence is equal to 1, then the unique lexical ID contained in the lexical ID sequence is added to the set of valid policy lexical IDs; If the length of the lexical ID sequence is not equal to 1, then the string corresponding to that lexical ID sequence is discarded.

[0011] Furthermore, the length of the mask vector is the same as the size of the vocabulary of the large language model. In the mask vector, the position corresponding to the lexical ID in the effective policy lexical ID set is configured as the allowed value, and the remaining positions are configured as the suppressed value; where the allowed value is 0 and the suppressed value is negative infinity.

[0012] Furthermore, step four specifically includes: In the generation process of each word, after obtaining the original logits vector L through the large language model, the mask vector M and the original logits vector L are added element-wise to obtain the masked logits vector L_masked, that is: ; in, This represents the logits vector after masking. Represents the original logits vector. The mask vector is represented by L; the logits vector L is a vector of length V, where each element represents the logarithm of the generation probability of the corresponding word. Will The input is fed into the Softmax function to calculate the final word probability distribution P; A preset sampling strategy is used to sample from the lexical probability distribution P to select the next lexical to be generated; wherein the preset sampling strategy includes, but is not limited to, a greedy sampling strategy or a Top-K sampling strategy.

[0013] Furthermore, it also includes dynamic policy update procedures, specifically: Real-time monitoring of changes to the financial credit policy codebook; When an update to the financial credit policy codebook is detected, steps one through three are automatically triggered and re-executed to update the set of valid policy term IDs and the mask vector.

[0014] Furthermore, it also includes: grouping the set of effective policy lexical IDs according to lexical type to construct submasks that correspond one-to-one with different credit parameter types; in step four, selecting the corresponding submask according to the currently generated context, and adding the corresponding submask to the original logits vector.

[0015] Furthermore, the term types include amount type, term type, and interest rate type.

[0016] Furthermore, it also includes error recovery steps: When the token sampling operation based on the masked logits vector in step four fails to sample a valid token that meets the preset semantic validity threshold for N consecutive times, a safety fallback mechanism is automatically triggered, where N is a preset positive integer, N≥1; the safety fallback mechanism includes: a) Record the context state information of the currently generated sequence. The context state information includes, but is not limited to, the semantic vector representations of the first K generated words and the corresponding attention weight distributions; where K is a preset positive integer, K≥5; b) Calculate the semantic offset parameter based on the context state information. The semantic offset parameter is obtained by comparing the KL divergence between the logits vector after the current mask and the logits vector in the historical normal generation stage. c) Adaptively select a target codebook from at least two preset security policy codebooks based on the semantic offset parameter. Each security policy codebook includes pre-trained security lexical sequence templates of different lengths and corresponding generation probability adjustment factors. When the semantic offset parameter is greater than a preset threshold T, the codebook that includes a longer security lexical sequence template is selected. The pre-trained security lexical sequence template is a fixed lexical sequence trained based on historical compliant credit texts. d) Use the target codebook to backgenerate the current generated sequence, specifically including: dynamically pruning the safe lexical sequence template in the target codebook based on context state information, and correcting the sampling probability of each lexical in the template by adjusting the generation probability factor, so as to generate a repaired lexical sequence with semantic coherence with the context greater than a preset threshold.

[0017] Furthermore, it also includes a policy codebook structure verification step, specifically: After obtaining the financial credit policy codebook in step one, a structured validation is performed on it, including: checking whether there are duplicate credit parameter value strings in the codebook; checking whether the format of each string conforms to the preset parameter type specifications, which include that amount type strings must contain numbers and currency units, term type strings must contain numbers and time units, and interest rate type strings must contain numbers and percentage signs; if there are duplicate strings or strings that do not conform to the specifications, a validation report is generated and the user is prompted to make corrections. After the corrections are made, the structured validation is re-executed until the validation passes.

[0018] Compared with existing technologies, this invention has the following advantages and beneficial effects: First, by using a policy mask constraint decoding mechanism, it ensures that the generated financial credit decision text strictly complies with policy regulations, avoiding potential violations in traditional methods and improving the compliance of decisions. Second, by using a pre-trained word segmenter to efficiently encode the policy codebook, combined with a dynamic filtering strategy, it effectively narrows the range of effective policy term IDs, improving the efficiency and accuracy of the decoding process. Third, by introducing a dynamic policy update step, it can respond in real time to changes in financial credit policies, automatically adjusting the effective policy term ID set and mask vector, ensuring that the system always remains consistent with the latest policies, enhancing the system's adaptability and flexibility. Fourth, by grouping the effective policy term ID set and constructing sub-masks corresponding to different credit parameter types, it achieves the function of dynamically selecting sub-masks based on the context, further improving the relevance and accuracy of the generated text. Fifth, the introduction of an error recovery step enables the system to automatically trigger a safety rollback mechanism when encountering generation difficulties. By calculating semantic offset parameters and adaptively selecting the target codebook, it effectively repairs errors in the generated sequence, ensuring the coherence and semantic consistency of the generated text. Sixth, the addition of a structured verification step for the policy codebook ensures the quality of the obtained financial credit policy codebook. By checking for duplicate strings and format specifications, potential problems are identified and corrected in advance, providing a reliable data foundation for the subsequent decoding process.

[0019] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram illustrating the steps of a RAG financial credit decision-making method based on policy mask constraint decoding. Figure 2 A schematic diagram illustrating the specific steps for further execution of the screening step in step three; Figure 3 This is a diagram illustrating the steps involved in updating dynamic policies. Detailed Implementation

[0022] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0023] This invention provides a RAG financial credit decision-making method based on policy mask constraint decoding, such as... Figure 1 As shown, it includes the following steps: Step 1: Obtain a structured financial credit policy codebook. The financial credit policy codebook includes string representations of multiple allowed credit parameter values. The structured financial credit policy codebook refers to a policy document in XML or JSON format that contains fields for parameter types and string values, facilitating batch encoding by a pre-trained word segmenter. Step 2: Initialize the set of valid policy term IDs; the set of valid policy term IDs is an empty set of integers used to store term IDs that have been verified and can be used for constraint decoding; Step 3: Using a pre-trained word segmenter matched with the large language model, encode each string in the financial credit policy codebook, filter out the strings that can be encoded into a single word, and add the word ID corresponding to the string to the set of effective policy word IDs; The large language model is a deep learning model trained on massive text data that can understand and generate human language. It usually contains billions or even hundreds of billions of parameters and can capture complex patterns and semantic information in language. It is widely used in tasks such as text generation, translation, and question answering. Step 4: During the decoding process of the large language model, the pre-constructed mask vector is added to the original logits vector output by the large language model to obtain the logits vector after the mask is applied. This forces the output lexical units to be constrained to the set of valid policy lexical unit IDs during the generation process, thereby realizing the generation of compliant financial credit decision text in a single decoding. In specific application scenarios, such as the approval of credit loans for micro and small enterprises, the large language model uses the LLaMA3-70B version, the pre-trained word segmenter is the official LLaMA3 word segmenter, and the hardware configuration is a GPU A100 (80GB VRAM). Assume a commercial bank has a credit loan policy for micro and small enterprises, where the allowed parameters for the enterprise's establishment period are: 1 year (inclusive) or more, 2 years (inclusive) or more, 3 years (inclusive) or more; the allowed parameters for the enterprise's annual revenue are: 500,000 RMB (inclusive) - 5,000,000 RMB (exclusive), 5,000,000 RMB (inclusive) - 20,000,000 RMB (exclusive), 20,000,000 RMB (inclusive) - 50,000,000 RMB (exclusive); and the allowed parameters for the credit limit are: 500,000 RMB, 1,000,000 RMB, 2,000,000 RMB, 3,000,000 RMB, 5,000,000 RMB. The specific implementation is as follows: Step 1: Obtain the structured financial credit policy codebook. In this scenario, the codebook includes the following strings representing the allowed credit parameter values: [1 year (inclusive) or more, 2 years (inclusive) or more, 3 years (inclusive) or more, RMB 500,000 (inclusive) - RMB 5,000,000 (exclusive), RMB 5,000,000 (inclusive) - RMB 20,000,000 (exclusive), RMB 20,000,000 (inclusive) - RMB 50,000,000 (exclusive), RMB 500,000, RMB 1,000,000, RMB 2,000,000, RMB 3,000,000, RMB 5,000,000]; Step 2: Initialize the set of valid strategy term IDs, initially as an empty set; Step 3: Using a pre-trained word segmenter matched with a large language model (e.g., assuming a certain version of the Llama model is used), encode each string in the financial credit policy codebook. First, encode strings with a duration of 1 year or more. Assuming the word segmenter encodes it as a single token, the resulting token ID is 10245, which is added to the set of valid policy token IDs. Next, encode strings with a duration of 2 years or more. Assuming the word segmenter encodes it as a single token, the resulting token ID is 10246, which is added to the set of valid policy token IDs. Then, encode strings with a duration of 3 years or more. The word segmenter encodes the string "500,000 yuan (inclusive) - 5,000,000 yuan (exclusive)" into a single word, resulting in word ID 10247, which is added to the set of valid strategy word IDs. Next, the word segmenter encodes the string "5,000,000 yuan (inclusive) - 5,000,000 yuan (exclusive)" into a single word, resulting in word ID 20567, which is added to the set of valid strategy word IDs. Then, the word segmenter encodes the string "5,000,000 yuan (inclusive) - 20,000,000 yuan (exclusive)" into a single word, resulting in word ID 20568, which is added to the set of valid strategy word IDs. Finally, the word segmenter encodes the string "20,000,000 yuan (inclusive) - 50,000,000 yuan (exclusive)" into a single word, resulting in word ID 20568, which is added to the set of valid strategy word IDs. Encoding the string "inclusive" as a single token, assuming the token segmenter encodes it as a single token, the resulting token ID is 20569, which is then added to the set of valid strategy token IDs. Encoding the string "500,000 yuan", assuming the token segmenter encodes it as a single token, the resulting token ID is 30123, which is then added to the set of valid strategy token IDs. Encoding the string "1,000,000 yuan", assuming the token segmenter encodes it as a single token, the resulting token ID is 30124, which is then added to the set of valid strategy token IDs. Encoding the string "2,000,000 yuan", assuming the token segmenter encodes it as a single token, the resulting token ID is 30125, which is then added to the set of valid strategy token IDs. 125 is added to the set of valid strategy lexical IDs; the string "3 million yuan" is encoded, assuming the word segmenter encodes it as a single word, resulting in word ID 30126, which is then added to the set of valid strategy lexical IDs; the string "5 million yuan" is encoded, assuming the word segmenter encodes it as a single word, resulting in word ID 30127, which is then added to the set of valid strategy lexical IDs; at this point, the set of valid strategy lexical IDs is {10245,10246,10247,20567,20568,20569,30123,30124,30125,30126,30127}; Step 4: During the large language model decoding process, the pre-constructed mask vector is added to the original logits vector output by the large language model to obtain the logits vector after masking. When constructing the mask vector, it is assumed that the vocabulary size of the large language model is 32000, and the mask vector is a vector of length 32000. For each position i in the mask vector: if i belongs to the set of valid policy token IDs {10245,10246,10247,20567,20568,20569,30123,3...} If i ∈ {0124,30125,30126,30127}, then the value of the mask vector at that position is 0 (or a very small positive value, which does not affect the probability distribution after Softmax); if i does not belong to the set of valid policy term IDs, then the value of the mask vector at that position is a very large negative number (e.g., -1e10); when inputting a query, for example, the user inputs the query: Based on the provided information of a certain micro-enterprise A (assuming that the RAG system has retrieved relevant information and compiled it into a prompt), what is the credit loan decision recommendation for this enterprise? The large language model receives a prompt containing information about the company and begins generating credit decision text. During decoding, when the model generates text that requires determining the company's establishment year, annual revenue, and corresponding credit parameters or credit limit, at each word prediction step, the model first calculates the original logits vector (32,000 elements, each element corresponding to the original score of a word). Then, it adds the pre-constructed mask vector element-wise to the original logits vector to obtain the masked logits vector. At this point, the logits values ​​corresponding to all ineffective policy word IDs become very small (because the original logits are increased by a very large negative number). A softmax operation is then performed on the masked logits vector to obtain the predicted probability of each word. At this point, only the probability corresponding to the effective policy word ID is not zero (or has a very high probability). The language model samples or selects the word with the highest probability based on this probability distribution as the currently generated word. For example, when describing the number of years a company has been established, the model can only select from words with IDs 10245, 10246, and 10247, i.e., generate one of 1 year (inclusive), 2 years (inclusive), or 3 years (inclusive). Similarly, when describing annual revenue and credit limit, it can only select from the strings specified in the policy codebook. In this way, the financial credit decision text generated by the large language model, for example: "Based on the situation of company A, its establishment period is 2 years (inclusive) or more, and its annual revenue is in the range of 5 million yuan (inclusive) to 20 million yuan (exclusive), and a credit limit of 2 million yuan is recommended." All credit parameter values ​​involved are strictly derived from the financial credit policy codebook, achieving compliance with regulations in a single decoding. In this embodiment, word IDs are examples, and the actual values ​​depend on the selected model and word segmenter.

[0024] The working principle of the above technical solution is as follows: By pre-constructing and screening a set of effective policy terminator IDs that meet the requirements of financial credit granting policies, a mask vector is used to intervene in the original logits vector output by the model during the decoding stage of the financial credit granting decision text generation in the large language model. Specifically, firstly, a structured financial credit granting policy codebook is obtained, which defines the string representation of the credit granting parameter values ​​that are allowed to be generated. Then, the set of effective policy terminator IDs is initialized, and the strings in the codebook are encoded using a pre-trained word segmenter matched with the large language model. Only the terminator IDs corresponding to those strings that can be encoded into a single terminator are selected and added to the set of effective policy terminator IDs, thereby ensuring that the selected terminator IDs can accurately correspond to the policy. The allowed credit parameter values ​​are determined by the policy. During the decoding process of the large language model, a pre-constructed mask vector is added to the original logits vector output by the model to form a masked logits vector. The mechanism of this mask vector is to suppress the logits values ​​corresponding to lexical IDs outside the effective policy lexical ID set (e.g., by setting extremely low values ​​or negative infinity), while not suppressing or promoting the logits values ​​corresponding to lexical IDs within the set. Thus, when the model generates each lexical, its probability distribution is forcibly constrained to the effective policy lexical ID set. Ultimately, this enables the large language model to directly generate decision text that complies with financial credit policy regulations in a single decoding process, achieving immediate compliance of the generated content.

[0025] The beneficial effects of the above technical solution are as follows: First, the solution provided in this embodiment improves the generation efficiency of financial credit decision text, effectively ensures the compliance of the generated content, enhances the transparency and interpretability of the decision-making process, has good adaptability and scalability, and can flexibly respond to policy changes in different financial institutions and different credit scenarios. It can quickly adapt to new policy requirements simply by updating the financial credit policy codebook.

[0026] In one embodiment, the credit granting parameter values include at least one of the following: credit amount range, credit term range, interest rate range, and necessary conjunctions for forming natural language sentences; the credit granting parameter values refer to various key numerical values or option sets used to describe and determine the specific terms of a credit granting scheme in a credit granting business; the credit amount range refers to dividing the credit amount into several preset, discrete intervals or fixed numerical levels, and each level represents a specific amount range or specific amount; the credit term range refers to dividing the credit term into several preset, discrete time period levels, and each level represents a specific term length (such as 1 month, 3 months, 6 months, etc.); the interest rate range refers to dividing the credit interest rate into several preset, discrete interest rate level grades, and each grade represents a specific interest rate percentage or interest rate interval; the necessary conjunctions refer to words used to connect sentence components (such as subject, predicate, object, attributive, adverbial, etc.) or different sentences during the process of natural language generation to ensure the integrity of the sentence structure, smooth semantics, and clear logic, such as "of", "is", "for", "and", "wherein", etc.

[0027] The working principle of the above technical solution is as follows: by setting the credit granting parameter values to include at least one of the credit amount range, credit term range, interest rate range, and necessary conjunctions for forming natural language sentences, when the system makes a credit granting decision or generates relevant credit granting information, it can perform standardized and hierarchical division and selection of the credit amount, credit term, and interest rate based on different preset ranges; at the same time, with the help of necessary conjunctions, the information elements corresponding to each selected credit granting parameter value range are organically combined according to the grammar rules of natural language, so as to automatically generate a natural language sentence that conforms to human language habits and contains specific credit granting conditions, in order to achieve clear, accurate, and natural presentation and transmission of credit granting information.

[0028] The beneficial effects of the above technical solution are as follows: adopting the solution provided in this embodiment can improve the efficiency and accuracy of credit granting decisions, and reduce errors caused by manual intervention and subjective judgment; at the same time, with the help of necessary conjunctions, the information elements corresponding to each credit granting parameter value range are organically combined, making the generated natural language sentences clearer, more accurate, and more natural, easier to understand and accept, which helps to improve the customer experience and enhance the risk control ability and market competitiveness of financial institutions.

[0029] In one embodiment, as Figure 2 shown, the screening step in step three further includes: Encoding operations are performed on a string to generate a sequence of lexical IDs consisting of one or more lexical IDs. Encoding operations refer to the process of converting the original string (text data) into a digital form (in this case, lexical IDs) that can be recognized and processed by a computer according to specific rules or algorithms. The purpose is to map natural language text to a finite, discrete symbol space to facilitate subsequent calculations and analysis. Determine if the length of the generated token ID sequence is equal to 1; a token refers to the smallest semantic unit into which text is segmented in natural language processing. A token can be a character, a word, a punctuation mark, or a sub-word segmented according to specific rules; a token ID is a unique numerical identifier assigned to each token after text tokenization. This is a key step in converting text data into numerical data that the model can process, and it is usually mapped based on a predefined vocabulary. If the length of the token ID sequence is equal to 1, then the unique token ID contained in the token ID sequence is added to the valid strategy token ID set. The token ID sequence is an ordered list of numbers formed by arranging multiple token IDs in the order in which they appear in the original string. The valid strategy token ID set is a data structure used to store token IDs that meet specific filtering strategies or conditions. The token IDs in the set are considered to be valid inputs or operation objects in subsequent processing. If the length of the lexical ID sequence is not equal to 1, then the string corresponding to that lexical ID sequence is discarded.

[0030] The working principle of the above technical solution is as follows: The input string is encoded to convert it into a sequence of one or more lexical IDs arranged in order. Then, the length of the generated lexical ID sequence is determined. If the length of the lexical ID sequence is exactly 1, it indicates that the string corresponds to only one independent lexical after encoding. In this case, the unique lexical ID in the sequence is selected and added to the set of valid strategy lexical IDs as a valid candidate for subsequent processing. If the length of the lexical ID sequence is not 1, i.e., the sequence contains multiple lexical IDs or is empty, the string is considered not to meet the preset filtering conditions, and its corresponding lexical ID sequence is discarded and not included in the set of valid strategy lexical IDs.

[0031] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, strings that meet specific conditions can be effectively filtered out and their corresponding unique token IDs can be included in the set of effective strategy token IDs, thereby providing more accurate and effective data support for subsequent financial credit decisions.

[0032] In one embodiment, the length of the mask vector is the same as the size of the large language model's vocabulary. In the mask vector, the position corresponding to the token ID in the effective policy token ID set is configured as an allowed value, and the remaining positions are configured as suppressed values; where the allowed value is 0 and the suppressed value is negative infinity. The mask vector, in deep learning, especially in natural language processing, is a vector used to mask or apply specific weights to certain elements. Its function is to indicate which elements in the model need to be focused on, retained, ignored, or suppressed during model computation by setting different values. In this scheme, the mask vector is used to control the large language model's selection permissions for different tokens in the vocabulary. The vocabulary refers to the set of all tokens used by the large language model during training and inference. Each token has a unique index (ID) in the vocabulary. The model's input and output are usually represented in the form of these token IDs. The size of the vocabulary refers to the total number of tokens in the set. The allowed value is the element value configured to allow the large language model to select the corresponding token ID. Here, the allowed value is set to 0, indicating that it will not negatively affect the predicted probability of the word ID, giving it a chance to be selected by the model. The suppression value refers to the element value that inhibits the large language model from selecting the corresponding word ID. Here, the suppression value is set to negative infinity. Its purpose is to make the probability of the corresponding word ID approach zero when the model performs probability calculations (such as through the Softmax function), so that it cannot be selected. The Softmax function is a commonly used activation function used to "compress" a K-dimensional vector containing arbitrary real numbers into another K-dimensional real vector, so that the value of each element is between (0,1), and the sum of all elements is 1. In classification problems or word prediction tasks, Softmax is often used to convert the output of the last layer of the model (usually called logits) into a probability distribution, representing the probability of each category or word being selected. When a certain logit value is negative infinity, its corresponding probability will approach zero after Softmax calculation.

[0033] The working principle of the above technical solution is as follows: By constructing a mask vector, the dimension of which is set to be equal to the total number of lexical units in the vocabulary used by the large language model, the element value at the index position corresponding to each lexical ID in the preset effective strategy lexical ID set in the vocabulary is configured as an allowed value (0 here), while all other element values ​​in the mask vector except for the corresponding positions are configured as suppressed values ​​(negative infinity here). After such configuration, the mask vector can apply selective constraints to the lexical units in the vocabulary when the large language model performs lexical prediction or generation: for lexical units in the effective strategy lexical ID set, since their corresponding mask position is an allowed value of 0, their prediction probability is not suppressed; for other lexical units in the vocabulary that are not included in the effective strategy lexical ID set, since their corresponding mask position is a suppressed value of negative infinity, their probability of being selected will be significantly suppressed, even approaching zero, after Softmax equal probability normalization, thereby achieving the technical effect of only allowing the selection of lexical units from the effective strategy lexical ID set for output.

[0034] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, it is possible to more accurately select keywords based on preset strategies during the financial credit decision-making process, avoid interference from invalid or policy-incompatible keywords, and improve the accuracy and compliance of the decision-making.

[0035] In one embodiment, step four specifically includes: In the generation process of each word, after obtaining the original logits vector L through the large language model, the mask vector M and the original logits vector L are added element-wise to obtain the masked logits vector L_masked, that is: ; in, This represents the logits vector after masking. Represents the original logits vector. The mask vector is represented by L; the logits vector L is a vector of length V, where each element represents the logarithm of the generation probability of the corresponding word. Will The input is fed into the Softmax function to calculate the final word probability distribution P; A preset sampling strategy is used to sample from the lexical probability distribution P to select the next lexical to be generated; wherein the preset sampling strategy includes, but is not limited to, a greedy sampling strategy or a Top-K sampling strategy.

[0036] The working principle of the above technical solution is as follows: In the process of generating sequences word by word in the big language model, for each word to be generated, the big language model first encodes and decodes the current context and outputs an original logits vector L. The dimension of the original logits vector L is the size of the vocabulary V. Each element value in the vector corresponds to the logarithm of the generation probability of a specific word in the vocabulary, which represents the original preference of the model for each candidate word in the current generation step. To guide or constrain the generation process of word elements, this embodiment introduces a mask vector M, which has the same dimension V as the original logits vector L. By performing element-wise addition on the mask vector M and the original logits vector L, i.e., L_masked = L + M, the masked logits vector L_masked is obtained. The essence of this operation is to adjust (enhance or weaken) the logits value corresponding to a specific word element based on the original logits vector L, thereby indirectly adjusting the proportion of these words elements in the subsequent probability distribution. Element-wise addition refers to the operation of adding the elements at corresponding positions of two vectors with the same dimension. For vectors L and M, each element of the resulting vector L_masked is L_masked[i] = L[i] + M[i]. Subsequently, the masked logits vector L_masked is input into the Softmax function. The Softmax function converts L_masked into a probability distribution P by exponentializing and normalizing each element of the input vector. Each element of P represents the probability that the corresponding word is selected as the next generated word in the current step, and the sum of all elements is 1. Finally, sampling is performed from the lexical probability distribution P according to a preset sampling strategy to determine the next actual generated lexical. The preset sampling strategy can be selected according to specific application requirements. For example, when using a greedy sampling strategy, the lexical with the highest probability value in probability distribution P is directly selected as the output; when using a Top-K sampling strategy, the top K lexicals with the highest probability values ​​in probability distribution P are first selected to form a candidate set, and then random sampling is performed from this candidate set based on their probability values. Through the above steps, the next lexical can be generated controllably while considering the original model output and external mask constraints. The lexical probability distribution refers to the vector obtained after processing by the Softmax function, where each element value represents the probability of the corresponding lexical being selected in the current generation step. The sampling strategy refers to the method of selecting a specific result from the probability distribution. The greedy sampling strategy directly selects the lexical with the highest probability value in the probability distribution as the output; the Top-K sampling strategy selects the top K lexicals with the highest probability from the probability distribution as a candidate set, and then randomly samples from these K candidate lexicals based on their probabilities. This method increases the diversity of generated results compared to greedy sampling.

[0037] The beneficial effects of the above technical solution are as follows: Using the solution provided in this embodiment, the generation results of the large language model can be more accurately guided and constrained according to policy requirements during the financial credit decision-making process. On the one hand, by introducing a mask vector to adjust the original logits vector, the generation probability of specific lexical units can be effectively controlled, making the generated text content more compliant with financial policy regulations, avoiding expressions that contradict policies, and improving the compliance of credit decisions. On the other hand, the preset multiple sampling strategies provide flexibility for the generation process. Furthermore, this method of decoding based on policy mask constraints can be closely integrated with the large language model, achieving effective control over the generated content without changing the basic architecture of the large language model, thus reducing the cost of model adaptation and adjustment.

[0038] In one embodiment, such as Figure 3 As shown, it also includes dynamic policy update steps, specifically: Real-time monitoring of changes to the financial credit policy codebook; When an update to the financial credit policy codebook is detected, steps one through three are automatically triggered and re-executed to update the set of valid policy term IDs and the mask vector. Change refers to any modification, addition, deletion, or adjustment of information contained in the financial credit policy codebook. Update refers to the occurrence of the objective fact that the financial credit policy codebook has been changed.

[0039] The working principle of the above technical solution is as follows: By monitoring the change status of the financial credit policy codebook in real time, when the system detects an update event of the financial credit policy codebook, it will automatically trigger a preset update mechanism. This mechanism will drive the system to re-execute the aforementioned steps one to three, that is, to re-extract strategy tokens, map strategy token IDs, and generate mask vectors, thereby achieving dynamic updates to the set of valid strategy token IDs and corresponding mask vectors to ensure consistency with the latest financial credit policy codebook. The preset update mechanism refers to the mechanism pre-set in the system to respond to updates to the financial credit policy codebook and execute subsequent actions. A series of program instructions or logical rules for updating operations; strategy lexical extraction refers to the process of identifying and extracting the smallest lexical units or phrases with independent semantic and decision-making significance from financial credit policy texts or codebooks; strategy lexical ID mapping refers to the process of associating extracted strategy lexical units with a unique identifier (ID) so that computer systems can store, identify and process them; mask vector generation refers to generating a vector composed of 0s and 1s (or other binary values) according to specific rules (such as the validity, importance or applicability of strategy lexical units in a specific scenario) to indicate whether the lexical unit at the corresponding position is enabled or considered.

[0040] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, it can be ensured that the financial credit decision-making system always operates based on the latest and most accurate policy information, avoiding decision-making deviations caused by lagging or incorrect policy information, thereby improving the accuracy and reliability of financial credit decisions; at the same time, the introduction of dynamic policy update steps enables the system to flexibly respond to frequent changes in financial credit policies, and can automatically complete the update of strategy terms and mask vectors without manual intervention, thereby improving the system's adaptability and operational efficiency.

[0041] In one embodiment, the method further includes: grouping the set of effective policy term IDs according to term type to construct sub-masks that correspond one-to-one with different credit parameter types; in step four, selecting the corresponding sub-mask based on the currently generated context, and adding the corresponding sub-mask to the original logits vector.

[0042] The working principle of the above technical solution is as follows: First, the effective strategy lexical ID set is grouped according to lexical type to more accurately correspond to different credit parameter types. Different credit parameter types play different roles in financial credit decision-making. By constructing sub-masks that correspond one-to-one through grouping, targeted processing of different types of parameters can be achieved. In step four, the corresponding sub-mask is selected based on the currently generated context. This is because the context can reflect the current decision-making situation and the parameter types to be considered. The corresponding sub-mask is added to the original logits vector. This allows for adjustment of the original logits vector according to different contexts and parameter types, making the final decision result more in line with the actual situation and policy requirements, thereby improving the scientificity and rationality of financial credit decision-making.

[0043] The beneficial effects of the above technical solution are as follows: The solution provided in this embodiment can effectively improve the accuracy and adaptability of financial credit decision-making. By constructing sub-masks corresponding to different types of credit parameters, the system can capture the specific impact of various parameters in the decision-making process in a more detailed manner, avoiding the problem of generalized parameter processing that may occur in traditional methods. At the same time, the method of dynamically selecting sub-masks and adjusting the original logits vector according to the above context enables the decision-making process to closely fit the actual business scenario, which not only follows policy norms but also fully considers the differences of individual customers, thereby enhancing the rationality and reliability of credit decision-making.

[0044] In one embodiment, the term type includes amount type, term type, and interest rate type.

[0045] The working principle of the above technical solution is as follows: When processing target text, the system first identifies and classifies each word element in the text. When a word element is identified as belonging to the amount type, the system determines it as an information unit representing a specific monetary quantity; when a word element is identified as belonging to the term type, the system determines it as an information unit representing a time interval or duration; when a word element is identified as belonging to the interest rate type, the system determines it as an information unit representing the ratio of interest to loan amount (principal) within a certain period. By clearly classifying word elements into the aforementioned amount, term, and interest rate types, the system can accurately extract and distinguish key business information of different dimensions in the target text, laying the foundation for subsequent data processing, analysis, or other business logic operations based on this information. The amount type refers to the information content carried by the word element belonging to a specific monetary quantity, usually containing a monetary unit or clearly inferred to be a monetary value; the term type refers to the information content carried by the word element belonging to the category of time length or time period, such as "year," "month," "day," and specific duration value combinations; the interest rate type refers to the information content carried by the word element belonging to the category of the ratio between interest and principal, usually presented in the form of a percentage or a specific interest rate unit.

[0046] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, the system can more accurately capture and process the core elements in financial credit decision-making, and improve the accuracy and efficiency of information processing.

[0047] In one embodiment, an error recovery step is also included: When the token sampling operation based on the masked logits vector in step four fails to sample a valid token that meets the preset semantic validity threshold for N consecutive times, a safety fallback mechanism is automatically triggered, where N is a preset positive integer, N≥1; the safety fallback mechanism includes: a) Record the context state information of the currently generated sequence. The context state information includes, but is not limited to, the semantic vector representations of the first K generated words and the corresponding attention weight distributions; where K is a preset positive integer, K≥5; b) Calculate the semantic offset parameter based on the context state information. The semantic offset parameter is obtained by comparing the KL divergence between the logits vector after the current mask and the logits vector in the historical normal generation stage. c) Adaptively select a target codebook from at least two preset security policy codebooks based on the semantic offset parameter. Each security policy codebook includes pre-trained security lexical sequence templates of different lengths and corresponding generation probability adjustment factors. When the semantic offset parameter is greater than a preset threshold T, the codebook that includes a longer security lexical sequence template is selected. The pre-trained security lexical sequence template is a fixed lexical sequence trained based on historical compliant credit texts. d) Use the target codebook to backgenerate the current generated sequence, specifically including: dynamically pruning the safe lexical sequence template in the target codebook based on context state information, and correcting the sampling probability of each lexical in the template by adjusting the generation probability factor, so as to generate a repaired lexical sequence with semantic coherence with the context greater than a preset threshold.

[0048] The working principle of the above technical solution is as follows: During sequence generation, when the system performs word sampling based on the masked logits vector in step four, if N consecutive samplings (N is a preset positive integer, N≥1) fail to obtain a valid word that meets the preset semantic validity threshold, a safety backoff mechanism is automatically activated to address semantic drift or generation blocking issues that may occur during the generation process. The preset semantic validity threshold is the critical value for judging the semantic matching degree between a word and its context, typically ranging from 0.6 to 0.8, and can be adjusted according to the financial credit scenario. The specific workflow of this safety backoff mechanism is as follows: First, in step a), the system records the contextual state information of the currently generated sequence. The contextual state information refers to the set of historical information related to the current generation position that can affect the generation of subsequent word units. It is the key for the model to understand the current context. This contextual state information includes, but is not limited to, the semantic vector representations of the first K generated word units and their corresponding attention weight distributions, where K is a preset positive integer and K≥5. This information together constitutes the basis for understanding the current generation context. Next, step b) is executed, and the semantic offset parameter is calculated based on the recorded context state information. This semantic offset parameter is obtained by calculating the KL divergence (Kullback-Leibler Divergence) between the logits vector obtained after the current masking operation and the logits vector in the historical normal generation stage (i.e., when the rollback mechanism is not triggered). KL divergence is a concept in information theory used to measure the difference between two probability distributions P and Q. Its value is non-negative. The more similar P and Q are, the smaller the KL divergence value is, and vice versa. Here, KL divergence is used to quantify the difference between the two probability distributions (obtained by transforming the logits vector through the Softmax function). The greater the difference, the higher the deviation of the current generation direction from the historical normal generation direction. Subsequently, step c) is executed, whereby a target codebook is adaptively selected from at least two preset security strategy codebooks based on the calculated semantic offset parameter. Each security strategy codebook contains pre-trained security lexical sequence templates of different lengths and corresponding generation probability adjustment factors. The selection rule is set as follows: when the semantic offset parameter is greater than the preset threshold T, the system will select the codebook containing the longer security lexical sequence template, in order to more effectively correct the large semantic offset by introducing a longer, verified security sequence. The preset threshold T is the critical value for judging the degree of semantic offset, which can be optimized based on historical data, for example, by obtaining the optimal value through verification of a large number of trust cases. Finally, step d) is executed, which involves backtracking the current generated sequence using the selected target codebook. Specifically, firstly, the safe lexical sequence template in the target codebook is dynamically pruned based on the context state information to ensure the relevance of the template to the current context. Then, the sampling probability of each candidate lexical in the template is corrected by applying the generation probability adjustment factor corresponding to the codebook. The ultimate goal is to generate a repaired lexical sequence. The semantic coherence of this sequence with the current context must be greater than a preset threshold, so that the sequence generation process can recover from the abnormal state and continue in a reasonable direction. Attention weight distribution refers to the degree of attention the model pays to each position of the lexical in the input sequence (or the generated sequence) when generating the current lexical.

[0049] The beneficial effects of the above technical solution are as follows: In the scenario of sequence generation for financial credit decision-making, when encountering abnormal situations such as semantic drift or generation blockage, the solution provided in this embodiment can quickly and effectively recover from errors, ensuring the semantic coherence and rationality of the generated results.

[0050] In one embodiment, a policy codebook structure verification step is also included, specifically: After obtaining the financial credit policy codebook in step one, a structured validation is performed on it, including: checking whether there are duplicate credit parameter value strings in the codebook; checking whether the format of each string conforms to the preset parameter type specifications, which include: amount type strings must contain numbers and currency units, term type strings must contain numbers and time units, and interest rate type strings must contain numbers and percentage signs; if there are duplicate strings or strings that do not conform to the specifications, a validation report is generated and the user is prompted to make corrections. After the corrections are made, the structured validation is re-executed until the validation passes.

[0051] The working principle of the above technical solution is as follows: After obtaining the financial credit policy codebook, a structured verification step is added to comprehensively check the credit parameter value strings in the codebook. First, it checks for duplicate strings to avoid data redundancy and potential logical errors. Second, it checks whether the format of each string conforms to the preset parameter type specifications. For example, amount type strings must contain numbers and currency units, term type strings must contain numbers and time units, and interest rate type strings must contain numbers and percentage signs. If duplicate strings or strings with non-compliant formats are found, the system will automatically generate a verification report and prompt the user to make corrections. After the user makes the corrections, the system will re-execute the structured verification until all strings conform to the specifications, ensuring the quality and accuracy of the codebook, thereby providing a reliable foundation for subsequent sequence generation.

[0052] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the quality of the financial credit policy codebook can be significantly improved, logical problems caused by data redundancy or format errors can be reduced, and the accuracy of the credit parameter value strings in the codebook can be ensured. This not only provides a solid and reliable data foundation for the subsequent sequence generation steps, but also greatly improves the accuracy and efficiency of the entire financial credit decision-making process. At the same time, through the automated structured verification and report generation mechanism, the cost and time of manual review can be reduced, and the level of automation of business processing can be improved.

[0053] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A RAG financial credit decision-making method based on policy mask constraint decoding, characterized in that, Includes the following steps: Step 1: Obtain a structured financial credit policy codebook, which includes string representations of multiple allowed credit parameter values; Step 2: Initialize the set of valid policy term IDs; the set of valid policy term IDs is an empty set of integers used to store the term IDs that have been verified and can be used for constraint decoding; Step 3: Using a pre-trained word segmenter that matches the large language model, encode each string in the financial credit policy codebook, filter out the strings that can be encoded into a single word, and add the word ID corresponding to the string to the set of effective policy word IDs; Step 4: During the decoding process of the large language model, the pre-constructed mask vector is added to the original logits vector output by the large language model to obtain the logits vector after the mask is applied. This forces the output lexical units to be constrained to the set of valid policy lexical unit IDs during the generation process, thereby realizing the generation of compliant financial credit decision text in a single decoding. It also includes error recovery steps: When the token sampling operation based on the masked logits vector in step four fails to sample a valid token that meets the preset semantic validity threshold for N consecutive times, a safety fallback mechanism is automatically triggered, where N is a preset positive integer, N≥1; the safety fallback mechanism includes: a) Record the context state information of the currently generated sequence. The context state information includes, but is not limited to, the semantic vector representations of the first K generated words and the corresponding attention weight distributions; where K is a preset positive integer, K≥5; b) Calculate the semantic offset parameter based on the context state information. The semantic offset parameter is obtained by comparing the KL divergence between the logits vector after the current mask and the logits vector in the historical normal generation stage. c) Adaptively select a target codebook from at least two preset security policy codebooks based on the semantic offset parameter. Each security policy codebook includes pre-trained security lexical sequence templates of different lengths and corresponding generation probability adjustment factors. When the semantic offset parameter is greater than a preset threshold T, the codebook that includes a longer security lexical sequence template is selected. The pre-trained security lexical sequence template is a fixed lexical sequence trained based on historical compliant credit texts. d) Use the target codebook to backgenerate the current generated sequence, specifically including: dynamically pruning the safe lexical sequence template in the target codebook based on context state information, and correcting the sampling probability of each lexical in the template by adjusting the generation probability factor, so as to generate a repaired lexical sequence with semantic coherence with the context greater than a preset threshold.

2. The RAG financial credit decision-making method based on policy mask constraint decoding according to claim 1, characterized in that, Credit parameters include at least one of the following: credit amount level, credit period level, interest rate level, and conjunctions used to form natural language sentences.

3. The RAG financial credit decision-making method based on policy mask constraint decoding according to claim 1, characterized in that, The screening steps in step three further include: Perform encoding operations on the string to generate a sequence of lexical IDs consisting of one or more lexical IDs; Determine if the length of the generated word ID sequence is equal to 1; If the length of the lexical ID sequence is equal to 1, then the unique lexical ID contained in the lexical ID sequence is added to the set of valid policy lexical IDs; If the length of the lexical ID sequence is not equal to 1, then the string corresponding to that lexical ID sequence is discarded.

4. The RAG financial credit decision-making method based on policy mask constraint decoding according to claim 1, characterized in that, The length of the mask vector is the same as the size of the vocabulary of the large language model. In the mask vector, the position corresponding to the lexical ID in the effective policy lexical ID set is configured as the allowed value, and the remaining positions are configured as the suppressed value; where the allowed value is 0 and the suppressed value is negative infinity.

5. The RAG financial credit decision-making method based on policy mask constraint decoding according to claim 1, characterized in that, Step four specifically includes: In the generation process of each word, after obtaining the original logits vector L through the large language model, the mask vector M and the original logits vector L are added element-wise to obtain the masked logits vector L_masked, that is: ， in, This represents the logits vector after masking. Represents the original logits vector. The mask vector is represented by L; the logits vector L is a vector of length V, where each element represents the logarithm of the generation probability of the corresponding word. Will The input is fed into the Softmax function to calculate the final word probability distribution P; A preset sampling strategy is used to sample from the lexical probability distribution P to select the next lexical to be generated; wherein the preset sampling strategy includes, but is not limited to, a greedy sampling strategy or a Top-K sampling strategy.

6. The RAG financial credit decision-making method based on policy mask constraint decoding according to claim 1, characterized in that, It also includes dynamic policy update steps, specifically: Real-time monitoring of changes to the financial credit policy codebook; When an update to the financial credit policy codebook is detected, steps one through three are automatically triggered and re-executed to update the set of valid policy term IDs and the mask vector.

7. The RAG financial credit decision-making method based on policy mask constraint decoding according to claim 1, characterized in that, It also includes: grouping the set of effective policy term IDs according to term type to construct sub-masks that correspond one-to-one with different credit parameter types; in step four, selecting the corresponding sub-mask based on the currently generated context, and adding the corresponding sub-mask to the original logits vector.

8. The RAG financial credit decision-making method based on policy mask constraint decoding according to claim 7, characterized in that, The term types include amount type, term type, and interest rate type.

9. The RAG financial credit decision-making method based on policy mask constraint decoding according to claim 1, characterized in that, It also includes a policy codebook structure verification step, specifically: After obtaining the financial credit policy codebook in step one, a structured validation is performed on it, including: checking whether there are duplicate credit parameter value strings in the codebook; checking whether the format of each string conforms to the preset parameter type specifications, which include that amount type strings must contain numbers and currency units, term type strings must contain numbers and time units, and interest rate type strings must contain numbers and percentage signs; if there are duplicate strings or strings that do not conform to the specifications, a validation report is generated and the user is prompted to make corrections. After the corrections are made, the structured validation is re-executed until the validation passes.

Images