Unstructured cue word optimization scoring method for intelligent credit audit assistant

Abstract

The invention discloses an unstructured cue word optimization scoring method for an intelligent credit audit assistant. The method comprises the steps of collecting an unstructured credit audit text of an enterprise, wherein the credit audit text comprises a credit audit material, a full-duty report and post-loan inspection document data; the collected credit audit text is partitioned and indexed, and an intelligent audit vector knowledge base is constructed; analyzing the credit audit text by using a large language model, generating audit suggestions for customer credit, and giving enterprise risk scores; associating the enterprise risk score with an actual default risk index, constructing a loss function for measuring the difference between the enterprise risk score and the actual default risk index, regarding a large model prompt as a training parameter, and performing back propagation iterative optimization prompt; and the enterprise risk score is visually displayed through a visual means. According to the method, the subjective problem in risk measurement can be effectively solved, the performance of the model in a task without a standard answer is improved, and a more reliable basis is provided for intelligent credit audit.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of financial risk control, and particularly relates to an unstructured prompt word optimization scoring method for an intelligent credit audit assistant. BACKGROUND

[0002] In the current credit risk management field, traditional scoring methods are facing multi-dimensional challenges, and their limitations mainly lie in three core aspects: the limited structured data dimension leads to a single risk assessment perspective, the key risk signals in unstructured text are not effectively captured, and the subjective judgment bias in the manual audit process is difficult to eliminate. These pain points jointly restrict the accuracy and efficiency of risk assessment in credit audit.

[0003] In view of the above problems, there are explorations of risk assessment based on natural language processing in the prior art. For example, through the semantic understanding ability of a large language model, text information vectorization and risk feature extraction are realized. Although the large language model has a certain semantic understanding ability, enterprise credit audit and risk scoring belong to a typical "no standard answer" task, and its rationality needs to be verified by the actual default of the enterprise. In the prior art, the prompt word design of the large language model relies on manual experience, and there is also a lack of effective verification of the rationality of its evaluation results, resulting in a deviation between the subjective evaluation results of the model on enterprise credit audit and the objective situation of the enterprise. SUMMARY

[0004] The purpose of the present application is to provide an unstructured prompt word optimization scoring method for an intelligent credit audit assistant to overcome the deficiencies in the prior art.

[0005] To achieve the above purpose, the present application provides an unstructured prompt word optimization scoring method for an intelligent credit audit assistant, comprising: Step 1, collecting credit audit materials, due diligence reports and post-loan inspection document data of enterprises; Step 2, block and index the collected data, and construct an intelligent audit vector knowledge base; Step 3, based on the intelligent audit vector knowledge base and using a large language model, analyzing the credit audit text of the current enterprise, generating an audit opinion on the client credit, and giving an enterprise risk score; Step 4, associating the enterprise risk score with the actual default risk indicator, constructing a loss function to measure the difference between the two, regarding the large model prompt as a training parameter, and iteratively optimizing the prompt through back propagation; Step 5, visually displaying the enterprise risk score through a visual means.

[0006] Further, the credit audit material includes the basic situation of the enterprise client, the pre-loan credit access situation, the loan withdrawal information, the early warning information, the overdue information, the post-loan collection situation and the lawsuit situation.

[0007] Further, the process of constructing the vector knowledge base in step 2 includes: Based on the hierarchical structure and segment semantics of the credit audit text, the document is processed by block, then the pre-trained text embedding model is used to convert the document block into a dense text vector with unified dimensions, and finally the generated document block vector is structured labeled according to the customer unified number, credit unified code and timestamp, and then stored in a vector database supporting efficient search.

[0008] Further, the step 3 specifically includes: Based on the prompt engineering paradigm to design prompt templates, let the large language model identify and extract risk points in enterprise credit, and form an audit report, and give the risk score of the enterprise based on the audit report .

[0009] Further, the loss function is: ; Wherein, is the output result of the loss function, is the actual default risk index of the enterprise, and the default loss rate is selected; represents the mean square error, which is used as a measurement index of the rationality of the enterprise risk score.

[0010] Further, the text gradient used to optimize the input prompt in step 4 is calculated by back propagation, and the gradient calculation method of the optimization process is as follows: ; Wherein, represents the first order partial derivative of the loss function to the risk score , represents the input prompt of the large language model, represents that the gradient is calculated and output by the large language model in the form of text, represents the optimization gradient of the input prompt.

[0011] Further, the input prompt is constantly iteratively optimized in combination with the text gradient and the existing input prompt: ; Wherein, represents the input prompt optimized in the th iteration, represents the input prompt optimized in the th iteration, Text gradient representing the optimization of the prompt word.

[0012] Further, a validation set val is set, and a stop criterion for prompt word optimization is set to prevent overfitting, as follows: ; wherein, is a new prompt word Loss on the validation set, is the current optimal loss set.

[0013] Further, the enterprise risk score is displayed through the score dashboard in step 5, and the trend curve of the scores in previous years is displayed to intuitively reflect the enterprise risk evaluation result.

[0014] Beneficial effects: 1. The present application combines a large amount of unstructured data in the field of credit audit (including but not limited to customer interview records, due diligence reports, post-loan inspection documents, etc.), realizes the evaluation of enterprise credit risk, and compared with the limitations of traditional methods relying on simple structured data, the application of multi-source unstructured data can help banks to more comprehensively assess the credit risk level of enterprises; 2. The present application uses a large language model (LLM) to process credit audit text, effectively compensates for the deficiency of traditional models in utilizing unstructured information, and in the process of designing prompt words, refers to the concept of prompt word engineering paradigm, uses role prompts, thought chain prompts and other methods to significantly improve the stability and accuracy of the output content of the large language model; 3. The present application associates the subjective evaluation result of the large language model with the real default index of the enterprise, constructs a loss function and back propagates, realizes the joint optimization of the risk index and the prompt word; this mechanism effectively solves the subjective bias problem caused by relying on artificial interpretation or fixed prompt words in traditional credit audit, makes the evaluation result of the bank on the credit risk of the enterprise highly match the actual default situation of the enterprise, and significantly improves the reliability of the evaluation result in the scene without standard answer. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a structural schematic diagram of the unstructured prompt word optimization scoring method for the intelligent credit audit assistant of the embodiment of the present application. DETAILED DESCRIPTION

[0016] The present application will be further illustrated below in combination with the drawings and specific embodiments, and the embodiments are implemented on the premise of the technical solution of the present application, and it should be understood that these embodiments are only used to illustrate the present application and not used to limit the scope of the present application.

[0017] As Figure 1 shown, the embodiment of the present application provides a unstructured prompt word optimization scoring method for an intelligent credit audit assistant, comprising: Step 1, collect the credit audit materials, due diligence reports and post-loan inspection document data of the enterprise. Among them, the credit audit materials include enterprise customer basic information, pre-loan credit access information, loan withdrawal information, early warning information, overdue information, post-loan collection information, legal action information and other data.

[0018] Step 2, block and index the collected data, and construct an intelligent audit vector knowledge base. Specifically, the credit audit text collected in step 1 is used as the core input, and a hybrid blocking method of "semantic coherence limit + fixed length bottom-up" is adopted. First, based on the natural paragraph structure of the document, preliminary splitting is carried out to ensure that each text block contains a complete semantic unit. For long paragraphs that exceed the preset length, further splitting is carried out according to punctuation marks such as periods and semicolons. For text blocks that still exceed the length, strictly cut according to the maximum length limit to ensure that the length of a single text block meets the requirements; further, a pre-trained text embedding model (such as Zhipu Embedding 3 or gte-Qwen2) is used to convert the text block into a vector; a vector database (such as Mivus, FAISS) supporting approximate nearest neighbor (ANN) search is selected as the storage carrier, and the vectorized text segments are stored in the vector database in the structure of "timestamp-client unified number-credit unified code-text block vector-original text block" in batches.

[0019] Step 3, based on the intelligent audit vector knowledge base and using a large language model, analyze the current enterprise's credit audit text, generate an audit opinion on the client's credit, and give an enterprise risk score. Specifically, based on the prompt engineering paradigm, a prompt template is designed to let the large language model identify and extract risk points in enterprise credit, and form an audit report, and based on the audit report, give an enterprise risk score . According to the unstructured prompt word, the cosine similarity is used to find relevant knowledge and data from the intelligent audit vector knowledge base, and the large model is based on the retrieved knowledge to output the audit opinion of the entire client credit according to the self-recurrence logic. The calculation method of cosine similarity is as follows: ; Where, is the embedding vector of the user's question; is the embedding vector of a certain one in the intelligent audit knowledge base; represents the modulus of the word vector in the high-dimensional semantic space.

[0020] Calculate and all cosine similarity, the Top N most relevant knowledge pieces are filtered out from the intelligent audit vector knowledge base, and the "user question + Top N relevant knowledge" is integrated into the enhanced prompt word. The enhanced prompt word needs to be encoded into "dynamic semantic representation" by the deep learning model architecture Transformer, that is, through capturing the contextual dependency relationship between word segmentation, a semantic vector strongly bound to the specific context is generated, which can accurately reflect the true semantics of the text in different scenarios. The final output of Transformer is the "probability distribution of the next word", and the model selects the word according to the probability, appends the new word to the context, and repeats the "modeling → generation" cycle until the complete credit approval opinion is output.

[0021] Step 4, associate the enterprise risk score with the actual default risk indicator, build a loss function to measure the difference between the two, and regard the large model prompt as a training parameter to iteratively optimize the prompt. Specifically, the risk score output by the large language model in step 3 is associated with the actual default risk indicator of the enterprise, the default loss rate is used as the core variable, and the mean square error is used to build the loss function: ; wherein, is the output result of the loss function, is the actual default risk indicator of the enterprise, and the default loss rate is selected; represents the mean square error, which is used as a measure of the rationality of the enterprise risk score. Specifically, The default loss rate refers to the amount of loss suffered by the bank or other financial institutions when the borrower defaults on the loan, expressed as a percentage of the total risk exposure at the time of default. The total amount of the financial institution is calculated using cumulative loss and risk exposure after evaluating all outstanding loans, and the formula is as follows: ; wherein, is the total amount of risk exposure, which refers to the total amount of outstanding exposure that the bank or financial institution assumes for the loan in the event of default. R is the total amount of recoverable funds, which refers to the total amount of funds actually recovered through the disposal of collateral, recovery, etc. after the loan defaults.

[0022] The input prompt word of the large language model is used as the parameter to be optimized, and the text gradient of the prompt word calculated by the loss function through back propagation is as follows: ; wherein, represents the first-order partial derivative of the loss function with respect to the risk score , represents the input prompt of the large language model, represents the gradient calculated by the large language model in the form of text output, represents the optimization gradient of the input prompt word.

[0023] The text gradient is back-propagated from the loss function layer to the prompt word layer, and the dimensions to be optimized in the prompt word are determined (such as focusing on the analysis dimension and refining the scoring standard granularity). Update the prompt word based on the text gradient: ; wherein, represents the input prompt word after the first iteration optimization, represents the input prompt word after the first iteration optimization, represents the input prompt word after the first iteration optimization, represents the input prompt word after the first iteration optimization, represents the text gradient of the optimized prompt word.

[0024] Also includes setting a validation set val, and setting the stopping criterion for prompt word optimization to prevent overfitting, as follows: ; wherein, is the loss of the new prompt word on the validation set, is the current optimal loss set.

[0025] Step 5, visually display the enterprise risk score through visual means. Specifically, the enterprise risk score can be displayed through a scoring dashboard, and the trend curve of the score over the years can be displayed to intuitively reflect the results of enterprise risk evaluation.

[0026] For example: XX Precision Machinery Co., Ltd. (hereinafter referred to as "XX Precision") is selected as a case enterprise. The selection is based on the general credit granting scenario for manufacturing enterprises in bank credit business, and strictly follows the "Enterprise Customer Basic Information" data collection standard, by setting specific background information and credit demand, to build an analysis sample close to the actual business of the bank.

[0027] Specific analysis process: 1. Data collection and preprocessing Basic business information constitutes the core basis for enterprise qualification evaluation, covering key elements such as registered capital and equity structure, which can intuitively reflect enterprise size and governance structure. The financial statements of the past three years serve as standardized financial data carriers, containing structured information such as balance sheets, income statements, and cash flow statements, providing data support for quantitatively analyzing enterprise solvency, profitability, and operational efficiency.

[0028] Synchronous collection of multiple types of unstructured data: Audit materials include basic information about enterprise clients, pre-loan credit access, loan withdrawal information, early warning information, overdue information, post-loan collection information, and legal action information, covering the entire life cycle of enterprise credit; Due diligence reports contain management interview records and industry risk analysis, which can be used to analyze the adaptability of enterprise strategic planning and industry environment; Post-loan inspection documents reflect the dynamic changes in the operating status of the enterprise through regular tracking records.

[0029] 2. Vector knowledge base construction and product description generation With text as input, the mixed block method of "semantic coherence + fixed length bottom" is used for text splitting. This method prioritizes the division of semantic units based on the internal logical structure of the document (such as chapter titles such as "loan withdrawal record" and "post-loan collection record"), while also using a fixed length constraint of 200-300 words as a bottom mechanism to ensure that the text blocks maintain semantic integrity and control the size of the data to adapt to the input requirements of the subsequent vectorization model.

[0030] Secondly, the Zhipu Embedding 3 model is used to convert the split text blocks into 768-dimensional dense vectors. The 768-dimensional vector dimension design balances the semantic expression ability and computational efficiency, allowing it to capture deep semantic features of the text while reducing resource consumption for subsequent retrieval and calculation through the dense vector format.

[0031] Finally, to achieve data traceability and efficient retrieval, a "timestamp - customer unified number - credit unified code" three-dimensional labeling structure is used to uniquely identify the vector data. Taking a case client as an example, the labeling format is "20250831-KH2025001-XD20250831", where the timestamp (20250831) records the data processing time, the customer number (KH2025001) is associated with the enterprise subject information, and the credit code (XD20250831) corresponds to the specific credit application project. The labeled vector data is stored in the Milvus vector database, and by building a structured vector index system dedicated to the case client, the unstructured data is transformed into a "semantic vector + structured metadata" dual-dimensional management mode, providing low-latency and high-relevance feature retrieval support for the credit scoring model.

[0032] 3. Generation of enterprise credit audit opinions and risk scoring Prompt word template design is the core link to ensure that the large model outputs results that meet the professional requirements of credit audit. The case-specific prompt word template integrates role positioning, task boundaries, and output specifications through structured design, providing a clear operational framework for the large model, thereby improving the accuracy and consistency of risk identification.

[0033] Role Definition Module: Clearly define the professional identity and core responsibilities of the large model, set as "You are a senior credit auditor of the bank, need to identify risk points based on the provided materials." This role positioning enables the model to simulate the thinking paradigm of professional auditors, focusing on key risk dimensions in credit evaluation and avoiding interference from non-professional perspectives. By anchoring the large model in the "senior auditor" role, it can effectively improve its understanding of complex financial data and industry information, ensuring that the analysis process meets the professional standards of bank credit auditing.

[0034] Task Instruction Module: Based on the thinking chain prompt paradigm, instruct the model to "extract risk points from solvency, operational stability, industry risk, and post-loan repayment enthusiasm, and label the severity (high / medium / low) and text block number for each risk point." This instruction ensures comprehensive and traceable analysis through three constraints: first, limit the core dimensions of risk assessment, covering the main sources of enterprise credit risk; second, require risk level quantification to avoid ambiguous statements; third, force the association of original text references to provide clear clues for subsequent manual review. This structured task design effectively reduces the randomness of model output, making the risk identification process repeatable.

[0035] Output Format Module: Specifies the standardized result presentation method, requiring a fixed format of "Audit Opinion: [Risk Point Description]; Risk Score (0-100 points): [Score]; Score Basis: [Text Block Reference]." This format achieves interpretability and comparability of results through three elements: risk point description needs to accurately summarize the problem nature, risk score uses a continuous 0-100 point scale to achieve fine-grained risk level differentiation, and text block reference ensures the traceability of the score result.

[0036] For the XX precision credit audit scenario, the system extracts key information from the vector knowledge base: "2024Q2 inspection report shows that 'the main supplier cooperation agreement will expire in January 2025'" and "the company has made multiple phone calls for collection without response," and inputs them into the large language model for analysis.

[0037] Based on semantic understanding and risk factor extraction of the input text, the model outputs structured audit opinions according to the industry risk assessment framework. Specifically, the analysis dimensions include solvency, operational stability, industry risk, and post-loan repayment enthusiasm. In the solvency dimension, by comparing the enterprise's current ratio (1.8) with the industry average (1.5), the risk level is determined to be "low". In the operational stability dimension, the model identifies that the expiration of core supplier cooperation may cause a risk of raw material supply disruption, and the risk level is rated as "medium". In the industry risk dimension, combined with the macro data of a 12% decline in downstream automobile manufacturing demand, the risk level is rated as "high". In the solvency dimension, the company has been repeatedly contacted for collection without response, and the risk level is rated as "high".

[0038] Based on multi-dimensional risk qualitative analysis, the model quantifies and integrates each risk by setting weights in the prompt words, and finally generates an initial risk score =68 points.

[0039] 4、Prompt word iterative optimization The actual default loss rate of this enterprise is 25%, corresponding to an actual risk score of 25 points. The mean square error (MSE) is used as the loss function. The initial output score of the model (68 points) and the actual risk score (25 points) are substituted into the MSE formula to calculate the loss function value.

[0040] Using the backpropagation principle, the initial prompt words are used as parameters to calculate the text gradient of the loss function on the prompt words through the large language model, accurately locating the source of the bias. The analysis results show that "insufficient industry risk weight" is the core factor leading to the bias of the model output, which means that the initial prompt words have not fully guided the model to conduct in-depth assessment of the industry environmental risk.

[0041] To address this issue, the large model automatically adjusts the prompt words to: "increase the 'industry demand fluctuation impact on enterprise income' analysis dimension, and require quantitative evaluation (such as 'every 1% decline in downstream demand, the enterprise's income is expected to decrease X%'). By introducing specific quantitative indicators, the model begins to directly link industry dynamics with enterprise financial performance, initially improving the accuracy of the assessment.

[0042] After 3 iterations of optimization, the final prompt words form a structured analysis framework: From the perspectives of solvency, operational stability, industry risk, and post-loan repayment willingness, the industry risk score weight is increased to 40%.

[0043] 5、Visual display of results The dashboard homepage highlights the customer's current real-time risk score of 48 points in red, which is significantly lower than the bank's safety threshold of 60 points, directly triggering the high-risk warning mechanism. This design ensures that auditors can quickly identify the risk level of the customer when logging into the system, providing priority guidance for further in-depth review.

[0044] The score change trajectory from 2022 to 2024 is displayed in the form of a line chart, showing that the customer's score has been continuously declining from 72 points in 2022 to 48 points in 2024, with a compound annual decline of 15%. The trend curve clearly presents the acceleration characteristics of risk deterioration: the decline rate was 16.7% (72→60) in 2022-2023, and further expanded to 20% (60→48) in 2023-2024, reflecting the increasing cumulative effect of risk factors.

[0045] The risk factor contribution pie chart visually displays the weight distribution of different core risk sources: industry risk accounts for 35%, supply chain risk accounts for 30%, debt repayment ability risk accounts for 25%, and post-loan repayment willingness risk accounts for 10%. Among them, industry risk as the primary contributing factor, indicates that auditors need to focus on the cyclical fluctuations of the industry to which the customer belongs, the influence of policy regulation, and changes in market competition patterns; supply chain risk points to the stability of the customer's upstream and downstream cooperation, which needs to be further verified by the supplier's dependence and logistics support capacity; debt repayment ability risk reflects the health of the enterprise's cash flow, which needs to be cross-verified with financial indicators such as current ratio and quick ratio; post-loan repayment willingness needs to be followed up by the collection personnel in real time.

[0046] The above is only the preferred embodiment of the present application, it should be noted that for ordinary skilled in the art, other not specifically described part, belongs to prior art or common knowledge. Without departing from the principles of the present application, can also make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. An unstructured cue optimization scoring method for intelligent credit audit assistant characterized in that, The application relates to a credit audit method and device. Step 1, collecting credit audit materials, due diligence reports and post-loan inspection document data of an enterprise; Step 2, block indexing of the collected data, and construction of an intelligent audit vector knowledge base; Step 3, based on the intelligent audit vector knowledge base, and using a large language model to analyze the current enterprise credit audit text, generating an audit opinion on the client credit, and giving an enterprise risk score; Step 4, associating the enterprise risk score with the actual default risk index, constructing a loss function to measure the difference between the two, regarding the large model prompt as a training parameter, and iteratively optimizing the prompt through back propagation; Step 5, visually displaying the enterprise risk score through a visual means.

2. The unstructured cue word optimization scoring method for intelligent credit audit assistant according to claim 1, wherein, The credit audit materials include enterprise client basic information, pre-loan credit access information, loan withdrawal information, early warning information, overdue information, post-loan collection information and lawsuit information.

3. The unstructured cue word optimization scoring method for intelligent credit audit assistant according to claim 1, wherein, The process of constructing the vector knowledge base in step 2 includes: Based on the hierarchical structure and segment semantics of the credit audit text, the document is block processed, then a pre-trained text embedding model is used to convert the document block into a vector, a dense text vector with unified dimensions is generated, finally the generated document block vector is structuredly labeled according to the customer unified number, credit unified code and timestamp, and is stored in a vector database supporting efficient search.

4. The unstructured cue word optimization scoring method for intelligent credit audit assistant according to claim 1, wherein, Step 3 specifically includes: Based on the prompt engineering paradigm, the prompt template is designed, the large language model is used to identify and extract risk points in enterprise credit, and an audit report is formed, and based on the audit report, a risk score of the enterprise is given .

5. The unstructured cue word optimization scoring method for intelligent credit audit assistant according to claim 1, wherein, The loss function is: ； Wherein, is the output result of the loss function, is the actual default risk index of the enterprise, and the selection is the default loss rate; represents the mean square error, which is a measure index of the rationality of the enterprise risk score.

6. The unstructured cue word optimization scoring method for intelligent credit audit assistant according to claim 5, wherein, In step 4, the text gradient used to optimize the input prompt word is calculated through back propagation, and the gradient calculation method of the optimization process is as follows: ； wherein, represents a first-order partial derivative of a loss function with respect to a risk score, represents an input prompt for a large language model, represents that the gradient is computed by the large language model in a text manner, represents an optimized gradient of the input prompt word.​ 7. The unstructured cue word optimization scoring method for intelligent credit audit assistant according to claim 6, wherein, Combine the text gradient with the existing input prompt word to iteratively optimize the input prompt word: ； wherein, represent the input prompt for the first iteration of the input prompt, represent the input prompt for the second iteration of the input prompt, represent the text gradient of the optimized prompt.

8. The unstructured cue word optimization scoring method for intelligent credit audit assistant according to claim 7, wherein, Also includes setting a validation set val and setting a stop standard for prompt word optimization to prevent overfitting, as follows: ； wherein, is a new prompt word loss on the validation set, is a set current optimal loss.

9. The unstructured cue word optimization scoring method for intelligent credit audit assistant according to claim 1, wherein, In step 5, the enterprise risk score is displayed through a score dashboard, the score trend curve of the past years is displayed, and the enterprise risk evaluation result is directly reflected.

Images