Risk decision-making method, electronic device, and computer-readable storage medium

Abstract

The application provides a risk decision method, an electronic device and a computer readable storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: inputting an input text sequence into a large language model, performing risk extraction processing on the input text sequence by the large language model using a first thought chain prompt template, obtaining a risk list, retrieving evidence information corresponding to each risk point from a heterogeneous field knowledge base based on risk information of each risk point, and determining an evidence confidence score of each risk point according to the evidence information of each risk point; updating the risk information of each risk point in the risk list based on the evidence confidence score of each risk point, and obtaining updated risk information of each risk point; determining a risk score of each risk point based on the updated risk information of each risk point, and determining a decision result according to the risk score of each risk point and the evidence confidence score corresponding to each risk point. The application avoids the model hallucination problem and improves the accuracy of the decision result.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a risk decision-making method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, risk control models based on machine learning and deep learning have been widely used in credit scoring, fraud detection, and loan decision-making. Existing risk control models mainly fall into two categories: one is the traditional statistical learning model, and the other is the semantic understanding model based on the Large Language Model (LLM). Both types of risk control models can output risk probabilities or decision results.

[0003] However, traditional statistical learning models are limited to numerical or categorical features in their decision-making process and cannot make decisions on unstructured or semi-structured natural language descriptions. Semantic understanding models based on large language models are prone to generating content that does not match the input facts during the decision-making process, resulting in a serious illusion problem and a lack of reasoning logic in the decision-making. Therefore, in practical applications, none of the above methods can guarantee the accuracy of the decision results. Summary of the Invention

[0004] The purpose of this application is to address the shortcomings of the prior art by providing a risk decision-making method, electronic device, and computer-readable storage medium to solve the problem that the prior art cannot guarantee the accuracy of decision results.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a risk decision-making method, the method comprising: Obtain the input text sequence, which is obtained by sequentially concatenating structured credit hard information, unstructured credit soft information, and user risk score; The input text sequence is input into a large language model, which uses the first thought chain prompt template to perform risk extraction processing on the input text sequence to obtain a risk list. The risk list includes multiple risk points and risk information for each risk point. Based on the risk information of each risk point, the evidence information corresponding to each risk point is retrieved from the heterogeneous domain knowledge base, and the evidence confidence score corresponding to each risk point is determined according to the evidence information corresponding to each risk point. The risk information of each risk point in the risk list is updated based on the evidence confidence score corresponding to each risk point, so as to obtain the updated risk information of each risk point. Based on the updated risk information of each risk point, the risk score of each risk point is determined, and the decision result is determined according to the risk score of each risk point and the corresponding evidence confidence score.

[0006] As one possible implementation, the first thought chain prompt template records a first instruction sequence, which includes a semantic recognition instruction, a rule matching instruction, an evidence backtracking and locating instruction, and a first output instruction arranged in sequence. The risk extraction process, which utilizes the first thought chain prompt template to extract risks from the input text sequence, yields a risk list, including: Execute the semantic recognition instructions to identify key risk signals from the input text sequence; Execute the rule matching instruction to determine whether the key risk signal triggers a predefined risk rule; If so, the evidence backtracking and positioning instruction is executed, the key risk signal is taken as the risk point, risk point description information is generated based on the description information corresponding to the risk point, and the risk type of the risk point and the location information of the risk point in the input text sequence are determined. Execute the first output instruction to use the risk type, risk point description information, and location information of the risk point as the risk information of the risk point, and store the risk point and the risk information of the risk point together in the risk list.

[0007] As one possible implementation, the step of retrieving evidence information corresponding to each risk point from a heterogeneous domain knowledge base based on the risk information of each risk point includes: The semantic similarity between multiple evidence entries stored in the heterogeneous domain knowledge base and the risk description information of the risk point is calculated respectively, and evidence entries with semantic similarity greater than a first preset threshold are selected as first evidence entries, and the score of the first evidence entry is determined according to the semantic similarity corresponding to the first evidence entry. Key features are extracted from the risk description information of the risk point, and the key features are matched with the key features of multiple evidence entries stored in the heterogeneous domain knowledge base to determine the feature matching degree between the risk description information of the risk point and each evidence entry. Evidence entries with a feature matching degree greater than a second preset threshold are selected as second evidence entries, and the score of the second evidence entry is determined according to the feature matching degree corresponding to the second evidence entry. Based on the scores of each first evidence item and each second evidence item, a predetermined number of evidence items are selected from the first and second evidence items as evidence items for the risk point. The evidence source of the evidence entry for the risk point is obtained from the heterogeneous domain knowledge base, and the evidence entry for the risk point and the evidence source of the evidence entry are used as the evidence information corresponding to the risk point.

[0008] As one possible implementation, determining the evidence confidence score corresponding to each risk point based on the evidence information corresponding to each risk point includes: Each risk point is compared with multiple evidence items corresponding to each risk point to determine the evidence status of each evidence item corresponding to each risk point. The evidence status includes confirmed status, conflicting status, or questionable status. Based on the evidence status of each evidence item, determine the evidence confidence score corresponding to each risk point.

[0009] As one possible implementation, updating the risk information of each risk point in the risk list based on the evidence confidence score corresponding to each risk point to obtain the updated risk information of each risk point includes: The evidence confidence score, evidence source, and evidence status of each evidence item corresponding to each risk point are added to the risk information of each risk point to obtain the updated risk information of each risk point.

[0010] As one possible implementation, determining the risk score of each risk point based on the updated risk information of each risk point includes: The large language model uses the second thinking chain prompt template to perform risk assessment on each risk point based on the updated risk information of each risk point, and determines the risk score of each risk point.

[0011] As one possible implementation, the second thought chain prompt template records a second instruction sequence, which includes a risk assessment instruction, a classification judgment instruction, an adjustment instruction, and a second output instruction arranged in sequence. The process of using the second thinking chain prompt template to perform risk assessment on each risk point based on the updated risk information of each risk point, and determining the risk score of each risk point, includes: Execute the aforementioned risk assessment instructions to determine the degree of impact of each risk point on the user's credit score; Execute the classification judgment instruction to determine the initial level of each risk point based on the degree of impact of each risk point on the user's credit score; Execute the adjustment instruction to adjust the initial level of each risk point based on the updated risk information of each risk point, thereby obtaining the severity level of each risk point; Execute the second output instruction to determine the risk score of each risk point based on the severity level of each risk point.

[0012] As one possible implementation, determining the decision result based on the risk score of each risk point and the corresponding evidence confidence score includes: The overall score for each risk point is determined based on the risk score of each risk point, the confidence score of the evidence corresponding to each risk point, the total number of risk points, and the risk score balance adjustment coefficient. The overall score of each risk point is mapped to obtain the rejection probability corresponding to each risk point; The decision result is determined based on the rejection probability corresponding to each risk point, and the decision result includes any one of the following: pass, reject, or review.

[0013] Secondly, embodiments of this application provide a risk decision-making device, the device comprising: The acquisition module is used to acquire the input text sequence, which is obtained by sequentially concatenating structured credit hard information, unstructured credit soft information, and user risk score; The risk extraction module is used to input the input text sequence into the large language model, and the large language model uses the first thinking chain prompt template to perform risk extraction processing on the input text sequence to obtain a risk list. The risk list includes multiple risk points and risk information for each risk point. The retrieval module is used to retrieve the evidence information corresponding to each risk point from the heterogeneous domain knowledge base based on the risk information of each risk point, and determine the evidence confidence score corresponding to each risk point based on the evidence information corresponding to each risk point. The update module is used to update the risk information of each risk point in the risk list based on the evidence confidence score corresponding to each risk point, so as to obtain the updated risk information of each risk point. The decision-making module is used to determine the risk score of each risk point based on the updated risk information of each risk point, and to determine the decision result based on the risk score of each risk point and the evidence confidence score corresponding to each risk point.

[0014] As one possible implementation, the first thought chain prompt template records a first instruction sequence, which includes a semantic recognition instruction, a rule matching instruction, an evidence backtracking and location instruction, and a first output instruction arranged in sequence; the risk extraction module is specifically used for: Execute the semantic recognition instructions to identify key risk signals from the input text sequence; Execute the rule matching instruction to determine whether the key risk signal triggers a predefined risk rule; If so, the evidence backtracking and positioning instruction is executed, the key risk signal is taken as the risk point, risk point description information is generated based on the description information corresponding to the risk point, and the risk type of the risk point and the location information of the risk point in the input text sequence are determined. Execute the first output instruction to use the risk type, risk point description information, and location information of the risk point as the risk information of the risk point, and store the risk point and the risk information of the risk point together in the risk list.

[0015] As one possible implementation, the retrieval module is specifically used for: The semantic similarity between multiple evidence entries stored in the heterogeneous domain knowledge base and the risk description information of the risk point is calculated respectively, and evidence entries with semantic similarity greater than a first preset threshold are selected as first evidence entries, and the score of the first evidence entry is determined according to the semantic similarity corresponding to the first evidence entry. Key features are extracted from the risk description information of the risk point, and the key features are matched with the key features of multiple evidence entries stored in the heterogeneous domain knowledge base to determine the feature matching degree between the risk description information of the risk point and each evidence entry. Evidence entries with a feature matching degree greater than a second preset threshold are selected as second evidence entries, and the score of the second evidence entry is determined according to the feature matching degree corresponding to the second evidence entry. Based on the scores of each first evidence item and each second evidence item, a predetermined number of evidence items are selected from the first and second evidence items as evidence items for the risk point. The evidence source of the evidence entry for the risk point is obtained from the heterogeneous domain knowledge base, and the evidence entry for the risk point and the evidence source of the evidence entry are used as the evidence information corresponding to the risk point.

[0016] As one possible implementation, the retrieval module is specifically used for: Each risk point is compared with multiple evidence items corresponding to each risk point to determine the evidence status of each evidence item corresponding to each risk point. The evidence status includes confirmed status, conflicting status, or questionable status. Based on the evidence status of each evidence item, determine the evidence confidence score corresponding to each risk point.

[0017] As one possible implementation, the update module is specifically used for: The evidence confidence score, evidence source, and evidence status of each evidence item corresponding to each risk point are added to the risk information of each risk point to obtain the updated risk information of each risk point.

[0018] As one possible implementation, the decision module is specifically used for: The large language model uses the second thinking chain prompt template to perform risk assessment on each risk point based on the updated risk information of each risk point, and determines the risk score of each risk point.

[0019] As one possible implementation, the second thought chain prompt template records a second instruction sequence, which includes a risk assessment instruction, a classification judgment instruction, an adjustment instruction, and a second output instruction arranged in sequence; the decision module is specifically used for: Execute the aforementioned risk assessment instructions to determine the degree of impact of each risk point on the user's credit score; Execute the classification judgment instruction to determine the initial level of each risk point based on the degree of impact of each risk point on the user's credit score; Execute the adjustment instruction to adjust the initial level of each risk point based on the updated risk information of each risk point, thereby obtaining the severity level of each risk point; Execute the second output instruction to determine the risk score of each risk point based on the severity level of each risk point.

[0020] As one possible implementation, the decision module is specifically used for: The overall score for each risk point is determined based on the risk score of each risk point, the confidence score of the evidence corresponding to each risk point, the total number of risk points, and the risk score balance adjustment coefficient. The overall score of each risk point is mapped to obtain the rejection probability corresponding to each risk point; The decision result is determined based on the rejection probability corresponding to each risk point, and the decision result includes any one of the following: pass, reject, or review.

[0021] Thirdly, embodiments of this application provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the risk decision-making method as described in any of the first aspects above.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the risk decision-making method as described in any of the first aspects above.

[0023] According to the risk decision-making method, electronic device, and computer-readable storage medium of this application, structured credit hard information, unstructured credit soft information, and user risk scores are concatenated into a unified input text sequence. This input text sequence is then input into a large language model, which uses a first thought chain prompt template to extract risks. This allows the large language model to identify multiple risk points and their risk information from the heterogeneous information of the input text sequence according to a clear reasoning path, thus alleviating the illusion problem caused by the lack of logical constraints in traditional large language models' direct decision-making. Based on this, for each identified risk point, corresponding evidence information is retrieved from a heterogeneous domain knowledge base, and an evidence confidence score is calculated based on the evidence information. This evidence confidence score is then used to update the risk information of the risk points in the risk list, i.e., external credible knowledge is used to verify and calibrate the risk points generated by the large language model. Finally, the risk score is determined based on the fused and updated risk information and the evidence confidence score, and the decision result is output. Thus, the embodiments of this application, through structured reasoning guided by thought chains and a fact verification mechanism enhanced by external knowledge retrieval, not only ensure the logic of the risk identification process, but also significantly reduce the possibility of the model generating false content out of thin air through objective evidence introduced by retrieval, thereby improving the accuracy of the decision results. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating a risk decision-making method provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating a risk list determination method provided in an embodiment of this application is shown; Figure 3 A flowchart illustrating an evidence information retrieval method provided in an embodiment of this application is shown. Figure 4 A flowchart illustrating a method for determining the risk score of a risk point according to an embodiment of this application is shown. Figure 5 A flowchart illustrating a decision result determination method provided in an embodiment of this application is shown; Figure 6 This paper shows a schematic diagram of the structure of a risk decision-making device provided in an embodiment of this application; Figure 7A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0027] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0028] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0029] Figure 1 A flowchart illustrating a risk decision-making method provided in an embodiment of this application is shown. (Refer to...) Figure 1 As shown, the method specifically includes the following steps: S101. Obtain the input text sequence.

[0030] Optionally, the input text sequence is obtained by sequentially concatenating structured hard credit information, unstructured soft credit information, and user risk score. The structured hard credit information, unstructured soft credit information, and user risk score are obtained by performing standardized preprocessing on three types of heterogeneous input sources. Specifically, the structured hard credit information is key-value pair feature data extracted from input sources such as credit reports and manually entered forms, for example, {"query_count_3m": 8, "monthly_income": 15000}, which is converted into a natural language descriptive paragraph using a predefined template, such as "This user has made 8 queries in the past 3 months, and their monthly income is 15,000 yuan." The unstructured soft credit information is unstructured rich text of credit information submitted by the user, such as debt descriptions and statements of occupational stability. The user risk score is a global risk score for a user output by an externally pre-trained risk control model. This risk score can be between 0 and 1000. The user risk score is converted into a key-value pair format as a reference threshold, such as "risk score of this application risk control model: [score value]".

[0031] Optionally, the above-mentioned structured credit information, unstructured credit information, and user risk scores can be sequentially concatenated to form an input text sequence.

[0032] S102. Input the input text sequence into the large language model. The large language model uses the first thinking chain prompt template to perform risk extraction processing on the input text sequence to obtain a risk list.

[0033] Optionally, the risk list includes multiple risk points and risk information for each risk point. A risk point is a specific hidden danger or anomaly identified from the input text sequence, while the risk information is a structured data set describing the risk point, consisting of three elements: risk type, risk point description information, and location information, used to uniquely identify and describe the risk point.

[0034] Optionally, after the input text sequence is input into the large language model, the large language model calls the first thought chain prompt template, executes semantic recognition instructions according to the reasoning logic defined by the first thought chain prompt template, identifies key risk signals such as abnormal load rate and high query density from the input text sequence, and compares the identified key risk signals with predefined risk rules to determine whether the key risk signal triggers the risk rule. If it is triggered, the key risk signal is identified as a risk point, the risk information of the risk point is determined, and the risk point and the risk information of the risk point are associated and stored, and a risk list in standard JSON format is output.

[0035] S103. Based on the risk information of each risk point, retrieve the evidence information corresponding to each risk point from the heterogeneous domain knowledge base, and determine the evidence confidence score corresponding to each risk point based on the evidence information corresponding to each risk point.

[0036] Optionally, embodiments of this application provide a dual-path parallel retrieval mechanism to retrieve evidence information corresponding to each risk point from a heterogeneous domain knowledge base. Specifically, the first path is a semantic similarity-based retrieval mechanism, used to filter semantically related first evidence entries from multiple evidence entries stored in the heterogeneous domain knowledge base and determine the score of the first evidence entries. The second path is a key feature-based precise matching mechanism, used to filter second evidence entries with high feature matching degrees from multiple evidence entries and determine the score of the second evidence entries. Then, the multiple first and second evidence entries obtained are uniformly sorted according to their scores from high to low, and a preset number of evidence entries with the highest ranking are selected as the final evidence entries for the risk points. Further, the original source information of each evidence entry for the finally determined risk points is retrieved from the heterogeneous domain knowledge base, and the evidence entry content is combined with the original source information of the evidence entries to obtain the complete evidence information corresponding to the risk points.

[0037] Optionally, each risk point is compared one by one with its corresponding multiple evidence items. Based on the comparison results, the evidence status of each evidence item is determined. This evidence status includes confirmed, conflicting, or questionable. Confirmed means the evidence supports the risk point; conflicting means the evidence contradicts the risk point or the risk point is a model illusion; and questionable means there is a lack of direct evidence. Based on this, a confidence weight is assigned to each evidence item according to its evidence status, and a weighted sum is calculated based on the confidence weight of each evidence item to obtain the evidence confidence score corresponding to each risk point.

[0038] S104. Update the risk information of each risk point in the risk list based on the evidence confidence score corresponding to each risk point, and obtain the updated risk information of each risk point.

[0039] Optionally, the evidence source of each evidence item corresponding to each risk point is obtained, and the evidence confidence score, evidence source and evidence status of each evidence item corresponding to each risk point are added to the risk information of each risk point to obtain the updated risk information of each risk point.

[0040] S105. Based on the updated risk information of each risk point, determine the risk score of each risk point, and determine the decision result based on the risk score of each risk point and the evidence confidence score corresponding to each risk point.

[0041] Optionally, the large language model uses the second thinking chain prompt template to analyze the specific impact of each risk point on the user's credit score, and determines the initial severity level of each risk point accordingly. Then, it dynamically corrects the initial level by combining the confidence weight of the evidence items and the user's original credit score, thereby determining the final severity level. By mapping the final determined severity level, the risk score of the risk point can be obtained.

[0042] Optionally, based on the risk score balance adjustment coefficient, the risk score of each risk point and the corresponding evidence confidence score are weighted and fused to calculate the comprehensive score of each risk point, so as to eliminate the model illusion of the large language model. Then, the calculated comprehensive score is mapped to the rejection probability, and the final decision result, including pass, reject or review, is output according to the rejection probability.

[0043] Based on this, the risk decision-making method according to the embodiments of this application concatenates structured credit hard information, unstructured credit soft information, and user risk scores into a unified input text sequence. This input text sequence is then fed into a large language model, which uses a first thought chain prompt template to extract risks. This allows the large language model to identify multiple risk points and their risk information from the heterogeneous information of the input text sequence according to a clear reasoning path, thus alleviating the illusion problem caused by the lack of logical constraints in traditional large language models' direct decision-making. Furthermore, for each identified risk point, corresponding evidence information is retrieved from a heterogeneous domain knowledge base, and an evidence confidence score is calculated based on this evidence information. This evidence confidence score is then used to update the risk information of the risk points in the risk list, i.e., external credible knowledge is used to verify and calibrate the risk points generated by the large language model. Finally, the risk score is determined based on the fused and updated risk information and the evidence confidence score, and the decision result is output. Thus, the embodiments of this application, through structured reasoning guided by thought chains and a fact verification mechanism enhanced by external knowledge retrieval, not only ensure the logic of the risk identification process, but also significantly reduce the possibility of the model generating false content out of thin air through objective evidence introduced by retrieval, thereby improving the accuracy of the decision results.

[0044] As one possible implementation, this application provides a dynamic routing strategy. After obtaining the input text sequence, dynamic path selection can be performed based on the word length and semantic complexity of the input text sequence. In this way, risk points are extracted by selecting different paths, thus balancing inference accuracy and response latency. The paths include lightweight paths and deep inference paths. A lightweight path refers to extracting risk points directly with a single prompt if the word length of the input text sequence is less than a preset threshold T and the input text sequence does not contain out-of-town credit reporting or complex overdue descriptions. A deep inference path refers to selecting a multi-level thought chain inference template and performing deep inference according to the thought chain template logic if the word length of the input text sequence is greater than or equal to the preset threshold T, or if the input text sequence involves multi-dimensional cross-data.

[0045] Optionally, using the first thought chain prompt template to perform risk extraction processing on the input text sequence indicates that a deep reasoning path has been selected. In this embodiment, the first thought chain prompt template records a first instruction sequence, which includes a semantic recognition instruction, a rule matching instruction, an evidence backtracking and location instruction, and a first output instruction arranged in sequence. After the input text sequence is input into the large language model, the large language model calls the pre-configured first thought chain prompt template to perform a risk extraction task, so as to identify and extract multiple risk points and the risk information of each risk point from the input text sequence, and associate and store each risk point and its risk information to obtain a risk list.

[0046] Figure 2 A flowchart illustrating a risk list determination method provided in an embodiment of this application is shown. (Refer to...) Figure 2 As shown, the above steps utilize the first thought chain prompt template to perform risk extraction processing on the input text sequence to obtain a risk list, specifically including the following steps: S201. Execute semantic recognition instructions to identify key risk signals from the input text sequence.

[0047] Optionally, the input text sequence includes unstructured credit information. Executing the semantic recognition instruction mainly involves identifying key risk signals from the unstructured credit information in the input text sequence. Specifically, the unstructured credit information is segmented into natural paragraphs to identify key risk signals, which include, but are not limited to: abnormal workload, high query density, overdue records, poor job stability, and hidden signals of multiple borrowing.

[0048] S202. Execute the rule matching instruction to determine whether the key risk signal triggers the predefined risk rule.

[0049] Optionally, for the identified key risk signals, a rule matching instruction is executed to determine whether the identified key risk signals trigger predefined risk rules, such as whether the debt-to-income ratio is ≥80% and if so, it is a heavy leverage risk, or whether the query frequency in the past 3 months is ≥8 times and if so, it is a high-frequency query risk.

[0050] S203. If so, execute the evidence backtracking and positioning instruction, take the key risk signal as the risk point, generate risk point description information based on the description information corresponding to the risk point, and determine the risk type of the risk point and the location information of the risk point in the input text sequence.

[0051] Optionally, if a key risk signal triggers a predefined risk rule, an evidence backtracking and location instruction is executed, treating the key risk signal as a risk point and recording the original text field or physical location of the risk point in the unstructured credit information. The corresponding original text field or physical location is formatted as "sentence Y of paragraph X" or "original text fragment containing the keyword", thereby determining the location information of the risk point in the input text sequence. This location information can be represented by precise positioning coordinates or in the form of a keyword index.

[0052] Optionally, the risk type of a risk point can be determined based on the risk rules triggered by the risk point, including but not limited to: high query density, excessive leverage ratio, etc. The descriptive information corresponding to the risk point refers to the original text field of the risk point in the unstructured credit reporting software information, and the descriptive information of the risk point can be generated based on the original semantics of the original text field of the risk point in the unstructured credit reporting software information.

[0053] S204. Execute the first output instruction, take the risk type, risk point description information and location information of the risk point as the risk information of the risk point, and store the risk point and the risk information of the risk point together in the risk list.

[0054] Optionally, the risk information for each risk point consists of three core fields: risk type, risk point description, and location information. After determining the risk type (risk_type), description, and evidence_position of a risk point, the first output instruction is executed to initialize an empty risk list. The determined risk points, along with their corresponding risk types (risk_type), descriptions, and evidence_positions, are then stored in the initialized risk list. Finally, the risk list is output in a standard JSON array format.

[0055] It is worth noting that after the input text sequence is fed into the large language model, the large language model loads the first thought chain prompt template and strictly follows the structured logic of the first thought chain prompt template to perform the risk extraction task, providing high-quality input for subsequent decision-making. For example, a high-level example structure of the first thought chain prompt template is shown below: "You are a senior retail credit risk control analyst. Please strictly follow these 4 steps:" Step A: Read the rich credit text section by section to identify all potential key risk signals... Step B: For each key risk signal, compare it one by one against the predetermined risk rules... Step C: Accurately trace back to the original text location for each risk point... Step D: Output the risk list in standard JSON array format only, without adding any extra text. The above example is a high-level example structure of the first thinking chain enhancement template. The four steps correspond to the four instructions contained in the first instruction sequence recorded in the first thinking chain prompt template.

[0056] Furthermore, the input text sequence also includes user risk scores. During the process of the Large Language Model (LLM) extracting risk points using the first thought chain prompt template, the user risk score plays a two-way verification and guidance role. Specifically, if the user risk score is in the high-risk range, for example, a user risk score ≥ 700, the LLM can receive additional instructions to prioritize and deeply analyze potentially hidden risk semantic regions in unstructured credit information, such as the implicit expression of "borrowing new money to repay old debts." Simultaneously, a feature anomaly deviation marking mechanism is established. Specifically, if the LLM extracts significant fraud signals from unstructured credit information but the original user risk score is low, the signal is marked as "feature anomaly deviation," and its weight is automatically increased.

[0057] In addition, this application provides a parallel processing strategy. If the input text sequence is too long, the input text sequence is divided into multiple semantic segments and the risk point extraction task is performed in parallel. Finally, deduplication and correlation integration are performed based on the segmentation information and semantic similarity, which can also generate a risk list.

[0058] Based on this, the embodiments of this application use a thought chain for deep reasoning to extract risk points, and break down the complex text analysis task into three sub-tasks: semantic recognition, rule matching, and evidence backtracking for distributed reasoning. This not only guides the large language model to accurately extract risk signals from unstructured credit information, but also verifies the basis for judging risk signals, effectively avoiding logical jumps or illusions that may occur when the large language model processes complex financial text, thereby significantly improving the accuracy of risk identification by the large language model.

[0059] Figure 3 A flowchart illustrating an evidence information retrieval method provided in an embodiment of this application is shown. (Refer to...) Figure 3 As shown, the above steps, based on the risk information of each risk point, retrieve the corresponding evidence information from a heterogeneous domain knowledge base, specifically including the following steps: S301. Calculate the semantic similarity between multiple evidence entries stored in the heterogeneous domain knowledge base and the risk description information of the risk point, and select evidence entries with semantic similarity greater than a first preset threshold as first evidence entries, and determine the score of the first evidence entry based on the semantic similarity corresponding to the first evidence entry.

[0060] Optionally, the heterogeneous domain knowledge base is a pre-built, multi-dimensional risk domain knowledge base, stored using a vector database with semantic embedding. The content of the heterogeneous domain knowledge base includes, but is not limited to: a financial risk control logic rule base, a historical default case base, a regulatory policy clause base, and an institution-specific blacklist and risk warning entries. Specifically, the financial risk control logic rule base includes institution-defined load ratios, query frequency thresholds, etc.; the historical default case base includes anonymized real default samples and attribution records, etc.; and the regulatory policy clause base includes relevant interim measures for credit management issued by the State Financial Supervision and Administration Bureau, etc.

[0061] Optionally, embodiments of this application provide a dual-path retrieval mechanism: one based on semantic similarity and the other based on feature matching. The semantic similarity-based retrieval mechanism utilizes a pre-trained semantic encoding model to transform the risk description information of risk points into high-dimensional query vectors. Simultaneously, it calculates the text vectors of multiple evidence entries stored in a heterogeneous domain knowledge base. Then, it quantifies the semantic closeness between the high-dimensional query vector corresponding to the risk description information of the risk point and the vectors of each evidence entry by calculating the cosine similarity. Only evidence entries with a cosine similarity value greater than or equal to a first preset threshold, such as 0.75, are retained, and these retained evidence entries are used as the first evidence entry. Further, the cosine similarity value corresponding to the first evidence entry can be directly used as the score of the first evidence entry.

[0062] S302. Extract key features from the risk description information of the risk point, perform feature matching between the key features and the key features of multiple evidence items stored in the heterogeneous domain knowledge base, determine the feature matching degree between the risk description information of the risk point and each evidence item, and select evidence items with a feature matching degree greater than a second preset threshold as second evidence items, and determine the score of the second evidence item based on the feature matching degree corresponding to the second evidence item.

[0063] Optionally, for the feature-matching retrieval mechanism, natural language processing techniques or regular expressions are used to extract highly identifiable key features from the risk description information of risk points, such as "number of overdue months" and "debt ratio." Then, the extracted key features are compared with the key features corresponding to multiple evidence entries stored in a heterogeneous domain knowledge base. The feature matching degree is calculated, and only evidence entries with a feature matching degree greater than a second preset threshold, such as 0.8, are retained as second evidence entries. Further, the feature matching degree corresponding to the second evidence entry can be directly used as the score of the second evidence entry.

[0064] S303. Based on the scores of each first evidence item and each second evidence item, select a predetermined number of evidence items from the first and second evidence items as evidence items for risk points.

[0065] Optionally, based on the multiple first evidence items and multiple second evidence items obtained through screening, the multiple first evidence items and multiple second evidence items are sorted from high to low according to the scores of each first evidence item and each second evidence item, and a preset number of evidence items are screened according to the sorting, such as screening the top 5 evidence items as evidence items of risk points.

[0066] It is worth noting that in the dual-path retrieval mechanism provided in this application, semantic retrieval based on cosine similarity can effectively capture the potential semantic connections between risk description information and evidence entries stored in heterogeneous domain knowledge bases, while precise matching based on key features ensures the accuracy of core risk control indicators. Furthermore, by sorting and filtering the dual-path retrieval results, not only is the comprehensiveness and relevance of evidence recall significantly improved, but reliable data support is also provided for subsequent risk assessment, effectively reducing credit risk caused by missing information or misjudgment.

[0067] S304. Obtain the evidence source of the evidence entries for risk points from the heterogeneous domain knowledge base, and use the evidence entries for risk points and the evidence source of the evidence entries as the evidence information corresponding to the risk points.

[0068] Optionally, after identifying the evidence entries for the risk points, the evidence sources for the evidence entries for the risk points are queried from the heterogeneous domain knowledge base. These evidence sources include the semantic index coordinates and hash check values ​​of the evidence entries in the original documents or credit reports. The evidence entries for the risk points and the evidence entries themselves are then identified as the evidence information corresponding to the risk points.

[0069] Based on this, the embodiments of this application provide a dual-path retrieval mechanism based on semantic similarity and feature matching degree, which is based on a pre-built heterogeneous domain knowledge base, and realizes in-depth mining and accurate positioning of risk evidence.

[0070] As one possible implementation, the above steps determine the evidence confidence score corresponding to each risk point based on the evidence information corresponding to each risk point, including: comparing each risk point with multiple evidence items corresponding to each risk point to determine the evidence status of each evidence item corresponding to each risk point, and determining the evidence confidence score corresponding to each risk point based on the evidence status of each evidence item.

[0071] Optionally, before determining the evidence confidence score corresponding to each risk point, it is necessary to ensure that each risk point is considered to be supported by evidence. In this application embodiment, a triple verification mechanism is provided to determine the authenticity and conflict of the extracted risk points. Only when one of the following three conditions is met is the risk point considered to be supported by evidence.

[0072] For example, the triple conditions include strong semantic correlation of vectors, precise matching of rule thresholds, and LLM (Large Language Model) auxiliary logic. Strong semantic correlation of vectors means that the risk point and the evidence item highly overlap in the vector space and have no numerical conflicts. Precise matching of rule thresholds means that the hard indicators involved in the risk point, such as being queried 8 times and verified by the rule base (query threshold is 6 times) to be a triggering risk with the highest priority. LLM auxiliary logic means that in cases of ambiguous boundaries, the large language model is invoked as a backup engine; the risk point and the retrieved evidence item are input, and the large language model outputs a "yes / no / partially supported" judgment.

[0073] Optionally, the evidence status includes confirmed, conflicting, or questionable. Confirmed status means that if the risk point description information is confirmed by retrieved evidence entries, or if, although there is no direct evidence, it conforms to the hard risk control rules defined by experts (e.g., the M1 overdue payment perfectly matches the fact base), the evidence status can be determined to be confirmed. Conflicting status includes two types: fraudulent conflict and logical elimination. If the risk point description information shows no overdue payment, but the retrieved evidence entries show overdue payment, then the user information disclosure is determined to be false, and the evidence status is determined to be conflicting. Logical elimination occurs when the risk point description information is confirmed by retrieval to be a semantic parsing error in the large language model (e.g., the risk description information contains no overdue payment statement, but the large language model generates a response description out of thin air), then the large language model is determined to be experiencing model illusion, and the evidence status is also determined to be conflicting. Questionable status means that if no evidence is found to directly support or refute the risk point, such as in very niche risk scenarios, the evidence status is determined to be questionable.

[0074] Optionally, if the evidence status of an evidence item is confirmed, the confidence weight of the evidence item is set to 1. If the evidence status of an evidence item is conflicting, specifically a fraudulent conflict as described above, the confidence weight of the evidence item is set to 1. However, if it is a logical elimination option, the confidence weight of the evidence item is set to 0. If the evidence status of an evidence item is questionable, the confidence weight of the evidence item is between 0.3 and 0.7, for example, 0.5. Based on this, the confidence weight of each evidence item is used as the confidence score of each evidence item. For each risk point, the confidence scores of multiple evidence items corresponding to each risk point can be weighted and summed to obtain the evidence confidence score corresponding to each risk point.

[0075] Based on this, the embodiments of this application introduce a multi-dimensional evidence status determination and differentiated weight allocation mechanism to achieve a quantitative assessment of the credibility of risk points.

[0076] As one possible implementation, step S104 above updates the risk information of each risk point in the risk list based on the evidence confidence score corresponding to each risk point, and obtains the updated risk information of each risk point, including: adding the evidence confidence score corresponding to each risk point, the evidence source and evidence status of each evidence item corresponding to each risk point to the risk information of each risk point, and obtaining the updated risk information of each risk point.

[0077] Optionally, the original risk list stores risk information for each risk point, including risk type (risk_type), risk point description (description), and location information (evidence_position). After determining the evidence confidence score (weight) for each risk point, the evidence source (evidence_source), and the evidence status (audit_mark) for each evidence item corresponding to each risk point, three new fields—evidence confidence score (weight), evidence source (evidence_source), and evidence status (audit_mark)—are added to the original fields of risk type (risk_type), risk point description (description), and location information (evidence_position) for each risk point. This updates the original risk information, resulting in updated risk information. At this point, the original risk list is also updated to a new risk list, containing more complete risk information for each risk point.

[0078] As one possible implementation, the above steps determine the risk score of each risk point based on the updated risk information of each risk point. This includes: using a large language model and a second thought chain prompt template, performing risk assessment processing on each risk point based on the updated risk information of each risk point, and determining the risk score of each risk point. The second thought chain prompt template records a second instruction sequence, which includes a risk assessment instruction, a grading judgment instruction, an adjustment instruction, and a second output instruction arranged in sequence.

[0079] Figure 4 A flowchart illustrating a method for determining the risk score of a risk point according to an embodiment of this application is shown. (Refer to...) Figure 4 As shown, the above steps utilize the second thinking chain prompt template to perform risk assessment on each risk point based on the updated risk information, and determine the risk score of each risk point. Specifically, the steps include the following: S401. Execute the risk assessment instruction to determine the degree of impact of each risk point on the user's credit score.

[0080] Optionally, user credit information is reflected in both the user's repayment ability and willingness to repay. Based on this, a risk assessment instruction is executed to evaluate the specific impact of each risk point on the user's repayment ability and willingness to repay. Among them, repayment ability focuses on quantitative indicators such as debt ratio and income stability, while repayment willingness focuses on behavioral signals such as inquiry frequency, job stability, and multiple borrowing.

[0081] S402. Execute the classification judgment instruction to determine the initial level of each risk point based on the degree of impact of each risk point on the user's credit score.

[0082] Optionally, a grading instruction is executed to classify the severity of each risk point, with severity levels including mild, moderate, and severe. The grades obtained after grading each risk point based on its impact on the user's credit score are only initial grades. Based on the reasoning logic of the second thinking chain template, the initial grades will be adjusted according to weights or scores to obtain a more accurate severity grading.

[0083] S403. Execute the adjustment instruction to adjust the initial level of each risk point based on the updated risk information of each risk point, and obtain the severity level of each risk point.

[0084] Optionally, after obtaining the initial level of each risk point, an adjustment instruction is executed to adjust the initial level of each risk point. Specifically, if the evidence item of a risk point is identified as isolated evidence or the confidence weight is less than 0.5, the severity level is reduced by one level based on the initial level. If the original user risk score is in the high-risk range, such as ≥700 points, the severity level of all identified risk points is increased by one level; if it is ≥800 points, it is increased by two levels.

[0085] S404. Execute the second output instruction to determine the risk score of each risk point based on the severity level of each risk point.

[0086] Optionally, a second output instruction is executed to map the severity level of each risk point to a risk score. Specifically, if the severity level is mild, the risk score is 1 point; if the severity level is moderate, the risk score is 3 points; and if the severity level is severe, the risk score is 5 points.

[0087] For example, the high-level example structure of the second thought chain prompt template is shown below: "You are now a bank credit approval manager with over 15 years of experience. Based on the updated risk list and the original user risk score, perform the following steps:" 1. Assess the specific impact of each risk factor on "repayment ability" and "repayment willingness"... 2. Classify the severity of each risk point (mild / moderate / severe) and explain the basis for the judgment... 3. Output structured intermediate results, using only JSON format, and do not include explanatory text. The above example is a high-level example structure of the second thinking chain enhancement template, corresponding to the multiple instructions contained in the second instruction sequence recorded in the second thinking chain prompt template.

[0088] Based on this, the embodiments of this application construct a deep reasoning mechanism from single-point risk identification to comprehensive risk assessment through the second thinking chain prompt template, thereby realizing the dynamic and accurate quantification of user credit risk.

[0089] Figure 5 A flowchart illustrating a decision outcome determination method provided in an embodiment of this application is shown. (Refer to...) Figure 5 As shown, the above steps determine the decision result based on the risk score of each risk point and the corresponding evidence confidence score, specifically including the following steps: S501. Determine the comprehensive score of each risk point based on the risk score of each risk point, the confidence score of the evidence corresponding to each risk point, the total number of risk points, and the risk score balance adjustment coefficient.

[0090] Optionally, in order to eliminate the computational illusion that may exist in large language models, after obtaining the risk scores of risk points, the comprehensive score of risk points is further determined based on the following formula (1): (1) in, This represents the overall score. This represents the risk score balance adjustment coefficient. Indicates the total number of risk points. Indicate risk points The risk score, This indicates the confidence score of the evidence corresponding to the risk point.

[0091] Furthermore, the risk score balance adjustment coefficient ,in This represents the user's risk score, a value between 0 and 1000.

[0092] S502. Map the comprehensive score of each risk point to obtain the rejection probability corresponding to each risk point.

[0093] Optionally, in this embodiment of the application, in order to meet the monotonicity requirement that the higher the comprehensive score, the higher the rejection probability, and to enable the intermediate risk range to have a higher resolution, which is more in line with the business needs of distinguishing boundary samples in risk control decision-making scenarios, this application uses an S-shaped probability mapping function to probabilize the comprehensive score of each risk point, so as to map the comprehensive score of each risk point to a rejection probability of 0%-100%.

[0094] For example, the overall score of each risk point is mapped to the rejection probability based on the probability mapping function shown in the following formula (2): (2) in, This represents the probability of rejection. This represents the overall score. This represents the mapping slope adjustment parameter, used to control the sensitivity of the rejection probability to changes in the overall score. This represents the mapping center parameter, which controls the score position where the rejection probability enters the rapidly rising range.

[0095] For example, the mapping slope adjustment parameter and mapping center parameters Pre-calibration can be performed based on historical credit samples, expert experience, or testing. As one possible implementation, in this embodiment, the mapping slope adjustment parameter... The possible value is 0.134, which is the mapping center parameter. A possible value is 4.392. Based on this, the overall score of the risk points... If the score is 6.8, then the corresponding rejection probability obtained from the mapping is 58%. If the overall score of the risk points... If the value is 22.61, then the rejection probability obtained by the corresponding mapping is 92%.

[0096] S503. Determine the decision result based on the rejection probability corresponding to each risk point.

[0097] The decision result includes any one of the following: pass, reject, or review.

[0098] Optionally, if the rejection probability is greater than 80%, or if there are at least two high-risk points, the decision result is determined to be rejection; if the rejection probability is between 40% and 80%, the decision result is determined to be review; and if the rejection probability is less than 40%, the decision result is determined to be approval.

[0099] Furthermore, after determining the decision outcome, the large language model outputs decision data containing semantic depth and numerical progression. This decision data is in JSON format and specifically includes the overall decision, rejection probability, and the array detailed_assessment. The detailed_assessment array contains risk points, severity levels, impact analysis on repayment ability and willingness, and risk scores.

[0100] Based on this, this application provides a dynamic weighted assessment mechanism based on a user's basic creditworthiness by introducing a risk score balancing adjustment coefficient, thereby achieving personalized calibration of risk assessment. In this way, not only is the comprehensive impact of each risk point quantified through weighted summation, but the adjustment coefficient also integrates the user's risk score into the decision-making process. This results in differentiated decision weights for the same risky behavior among users with different credit profiles, thus avoiding risk control misjudgments and ensuring the accuracy of decision results.

[0101] As a possible implementation, to ensure the logical rigor of the large language model, this application mandates threshold constraint checks and semantic consistency verification before generating explanations. For example, a high rejection probability must correspond to a high-risk point, effectively preventing mismatches between conclusions and reasons. Addressing the common illusion problem in models, this application also introduces reinforcement learning or direct preference alignment techniques based on human feedback. It utilizes tens of thousands of samples annotated by senior credit review experts to fine-tune the large language model, focusing on correcting causal inversions and fabricated evidence, thus keeping the factual illusion rate extremely low.

[0102] The risk decision-making method provided in this application will be explained and illustrated through specific embodiments below: Example 1: High-risk rejection case Structured credit information: 12 inquiries in the past 3 months, monthly income of 8,000 yuan, debt-to-income ratio of 75%, and one M1 delinquency. Unstructured credit information: "Due to recent business cash flow needs, I have borrowed money from multiple online lending platforms to repay previous loans. My total debt is currently approximately 60,000 yuan, and my monthly income is stable at around 8,000 yuan. I have no other negative records." User risk score: 780, in the high-risk range. Accordingly, the structured credit information is converted into natural language paragraphs and concatenated with unstructured credit information. Then, the phrase "Risk score of this application's risk control model: 780" is appended to the end, resulting in the input text sequence. Because the word length of the input text sequence exceeds the preset threshold T and the user's risk score is in the high-risk range, a deep reasoning path is triggered. The Large Language Model (LLM) calls the first thought chain prompt template and executes a logical reasoning process of "segment-by-segment reading, rule matching, evidence backtracking, and structured JSON output" to obtain an initial risk list. The JSON format of this list is shown below: [ { "risk_type": "high query density", "description": "12 inquiries in the past 3 months, far exceeding the normal level, indicating multiple borrowing behavior," "evidence_position": "Structured features + rich text paragraph 1" }, { "risk_type": "Hidden risk of using new money to repay old debts", "description": "The loan was used to repay previous loans, raising suspicions of rolling over loans." "evidence_position": "The first sentence in the rich text, 'Borrowed money from multiple online lending platforms to repay previous loans'" }, { "risk_type": "Excessive leverage", "description": "Debt-to-income ratio of 75%, approaching the threshold for severe leverage", "evidence_position": "Structured features" } ] For each risk point, a dual-path parallel retrieval mechanism is executed to retrieve the top 5 evidence entries from a heterogeneous domain knowledge base, as detailed below: High query density: Vector similarity 0.92, matching rule "≥8 queries in the last 3 months", confirmed state, confidence weight W=1.0, evidence_source="institutional risk control rule base + historical A / B test data".

[0103] Implicit risks of refinancing: Vector similarity 0.88, matching historical default cases of "refinancing pattern", confirmed state, W=1.0, audit_mark="confirmed".

[0104] Excessive leverage ratio: Matching rule "debt-to-income ratio ≥ 70% is considered leverage risk", W=0.9.

[0105] Furthermore, the risk information is updated by adding the fields weight, evidence_source, and audit_mark. Then, the second thought chain prompt template is invoked to execute the following reasoning logic: Factor assessment: High query density mainly affects "repayment willingness", while refinancing affects "repayment ability + willingness"; Severity classification: The first two categories are classified as "severe" (rule-driven + global score adjustment increases by one level), and the third category is classified as "moderate". The deterministic formula is calculated as follows: Score_total = 1.78 × (5 × 1.0 + 5 × 1.0 + 3 × 0.9) = 1.78 × 12.7 ≈ 22.61, the mapped rejection probability is approximately 92%, and if the mapped rejection probability is greater than 80% and there are 2 high-risk points, then the output decision result is rejection.

[0106] Furthermore, all intermediate results generated during the decision-making process are combined to ultimately output the following standard JSON data for direct use in the credit review system, regulatory reporting, and user notification: { "overall_decision": "Rejected", "risk_score": 780, "final_reason": "Based on a comprehensive assessment, the applicant has engaged in multiple borrowing activities (12 inquiries in the past 3 months) and carries the hidden risk of using new loans to repay old ones, with a high debt-to-income ratio. The overall rejection probability is 92%. The main risk basis is the credit inquiry record and the paragraph in the applicant's statement that 'the loan was used to repay previous loans.'" "key_risk_points": [ { "risk_type": "high query density", "description": "Queryed 12 times in the past 3 months...", "evidence_source": "Credit Report + Risk Control Rule Base", "severity": "severe" "confidence_weight": 1.0 }, / / ... other risk points ] } Example 2: Case Study of Manual Boundary Verification Applicant: Monthly income 12,000 yuan, 4 inquiries, debt ratio 45%, rich text includes "occasionally uses credit card for cash flow". Original risk score: 620 points.

[0107] Two low-risk points are extracted, one corresponding to a confirmed state and the other to a questionable state. The overall score is calculated to be Score_total≈6.8, with a rejection probability of 58%, which falls within the 40%-80% range corresponding to manual review. Therefore, the output decision is manual review, and low-confidence risk points are marked for manual attention. Finally, JSON format data is generated and clearly marked "It is recommended to manually verify the occupational stability description".

[0108] Based on the same inventive concept, this application also provides a risk decision-making device corresponding to the risk decision-making method. Since the principle of the risk decision-making device in this application is similar to the risk decision-making method described above, the implementation of the risk decision-making device can refer to the implementation of the risk decision-making method, and the repeated parts will not be described again.

[0109] Reference Figure 6 The diagram shown is a structural schematic of a risk decision-making device provided in an embodiment of this application. The risk decision-making device 600 includes: an acquisition module 601, a risk extraction module 602, a retrieval module 603, an update module 604, and a decision-making module 605, wherein: The acquisition module 601 is used to acquire the input text sequence, which is obtained by sequentially concatenating structured credit hard information, unstructured credit soft information, and user risk score; The risk extraction module 602 is used to input the input text sequence into the large language model. The large language model uses the first thinking chain prompt template to perform risk extraction processing on the input text sequence to obtain a risk list. The risk list includes multiple risk points and risk information for each risk point. The retrieval module 603 is used to retrieve the evidence information corresponding to each risk point from the heterogeneous domain knowledge base based on the risk information of each risk point, and to determine the evidence confidence score corresponding to each risk point based on the evidence information corresponding to each risk point. The update module 604 is used to update the risk information of each risk point in the risk list based on the evidence confidence score corresponding to each risk point, so as to obtain the updated risk information of each risk point. The decision module 605 is used to determine the risk score of each risk point based on the updated risk information of each risk point, and to determine the decision result based on the risk score of each risk point and the evidence confidence score corresponding to each risk point.

[0110] Based on this, the risk decision-making device according to the embodiments of this application concatenates structured credit hard information, unstructured credit soft information, and user risk scores into a unified input text sequence. After inputting the input text sequence into a large language model, the large language model calls the first thought chain prompt template to extract risks. This enables the large language model to identify multiple risk points and their risk information from the heterogeneous information of the input text sequence according to a clear reasoning path, thereby alleviating the illusion problem caused by the lack of logical constraints when traditional large language models make direct decisions. On this basis, for each identified risk point, corresponding evidence information is further retrieved from a heterogeneous domain knowledge base, and an evidence confidence score is calculated based on the evidence information. The evidence confidence score is then used to update the risk information of the risk points in the risk list, that is, external credible knowledge is used to verify and calibrate the risk points generated by the large language model. Finally, the risk score is determined and the decision result is output based on the fused and updated risk information and the evidence confidence score. Thus, the embodiments of this application, through structured reasoning guided by thought chains and a fact verification mechanism enhanced by external knowledge retrieval, not only ensure the logic of the risk identification process, but also significantly reduce the possibility of the model generating false content out of thin air through objective evidence introduced by retrieval, thereby improving the accuracy of the decision results.

[0111] In one possible implementation, the first thought chain prompt template records a first instruction sequence, which includes a semantic recognition instruction, a rule matching instruction, an evidence backtracking and location instruction, and a first output instruction arranged in sequence; the aforementioned risk extraction module 602 is specifically used for: Execute semantic recognition instructions to identify key risk signals from the input text sequence; Execute rule matching instructions to determine whether key risk signals trigger predefined risk rules; If so, execute the evidence backtracking and positioning instruction, take the key risk signal as the risk point, generate risk point description information based on the description information corresponding to the risk point, and determine the risk type of the risk point and the location information of the risk point in the input text sequence; Execute the first output instruction to use the risk type, risk point description information, and location information of the risk point as the risk information of the risk point, and store the risk point and the risk information of the risk point together in the risk list.

[0112] In one possible implementation, the retrieval module 603 is specifically used for: The semantic similarity between multiple evidence entries stored in the heterogeneous domain knowledge base and the risk description information of the risk point is calculated respectively. Evidence entries with semantic similarity greater than a first preset threshold are selected as first evidence entries. The score of the first evidence entry is determined according to the semantic similarity corresponding to the first evidence entry. Key features are extracted from the risk description information of the risk point. The key features are matched with the key features of multiple evidence items stored in the heterogeneous domain knowledge base to determine the feature matching degree between the risk description information of the risk point and each evidence item. Evidence items with a feature matching degree greater than the second preset threshold are selected as second evidence items. The score of the second evidence item is determined according to the feature matching degree corresponding to the second evidence item. Based on the scores of each first evidence item and each second evidence item, a predetermined number of evidence items are selected from the first and second evidence items as evidence items for risk points. The evidence sources for risk points are obtained from heterogeneous domain knowledge bases, and the evidence entries for risk points and their evidence sources are used as the evidence information corresponding to the risk points.

[0113] In one possible implementation, the retrieval module 603 is specifically used for: Each risk point is compared with the multiple evidence items corresponding to each risk point to determine the evidence status of each evidence item corresponding to each risk point. The evidence status includes confirmed status, conflicting status, or questionable status. Based on the evidence status of each evidence item, determine the evidence confidence score corresponding to each risk point.

[0114] In one possible implementation, the update module 604 is specifically used for: The evidence confidence score, evidence source, and evidence status of each evidence item corresponding to each risk point are added to the risk information of each risk point to obtain the updated risk information of each risk point.

[0115] In one possible implementation, the decision module 605 is specifically used for: The large language model uses the second thinking chain prompt template to perform risk assessment on each risk point based on the updated risk information of each risk point, and determines the risk score of each risk point.

[0116] In one possible implementation, the second thought chain prompt template records a second instruction sequence, which includes a risk assessment instruction, a classification judgment instruction, an adjustment instruction, and a second output instruction arranged in sequence; the aforementioned decision module 605 is specifically used for: Execute risk assessment instructions to determine the extent of the impact of each risk point on the user's credit score; Execute the classification judgment instruction and determine the initial level of each risk point based on the degree of impact of each risk point on the user's credit score; Execute the adjustment command to adjust the initial level of each risk point based on the updated risk information of each risk point, and obtain the severity level of each risk point; Execute the second output instruction to determine the risk score of each risk point based on the severity level of each risk point.

[0117] In one possible implementation, the decision module 605 is specifically used for: The overall score for each risk point is determined based on the risk score of each risk point, the confidence score of the evidence corresponding to each risk point, the total number of risk points, and the risk score balance adjustment coefficient. The overall score of each risk point is mapped to obtain the rejection probability corresponding to each risk point; The decision result is determined based on the rejection probability corresponding to each risk point. The decision result includes any one of the following: pass, reject, or review.

[0118] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0119] This application also provides an electronic device 700, such as... Figure 7 The diagram shown is a structural schematic of an electronic device 700 provided in an embodiment of this application, including: a processor 701 and a memory 702, and optionally, a bus 703. The memory 702 stores machine-readable instructions executable by the processor 701. When the electronic device 700 is running, the processor 701 and the memory 702 communicate via the bus 703. When the machine-readable instructions are executed by the processor 701, the steps of the risk decision-making method described in any of the preceding claims are performed.

[0120] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the risk decision-making method described in any of the preceding claims.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0122] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0123] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A risk decision-making method, characterized in that, include: Obtain the input text sequence, which is obtained by sequentially concatenating structured credit hard information, unstructured credit soft information, and user risk score; The input text sequence is input into a large language model, which uses the first thought chain prompt template to perform risk extraction processing on the input text sequence to obtain a risk list. The risk list includes multiple risk points and risk information for each risk point. Based on the risk information of each risk point, the evidence information corresponding to each risk point is retrieved from the heterogeneous domain knowledge base, and the evidence confidence score corresponding to each risk point is determined according to the evidence information corresponding to each risk point. The risk information of each risk point in the risk list is updated based on the evidence confidence score corresponding to each risk point, so as to obtain the updated risk information of each risk point. Based on the updated risk information of each risk point, the risk score of each risk point is determined, and the decision result is determined according to the risk score of each risk point and the corresponding evidence confidence score.

2. The method according to claim 1, characterized in that, The first thought chain prompt template records a first instruction sequence, which includes a semantic recognition instruction, a rule matching instruction, an evidence backtracking and locating instruction, and a first output instruction arranged in sequence. The risk extraction process, which utilizes the first thought chain prompt template to extract risks from the input text sequence, yields a risk list, including: Execute the semantic recognition instructions to identify key risk signals from the input text sequence; Execute the rule matching instruction to determine whether the key risk signal triggers a predefined risk rule; If so, the evidence backtracking and positioning instruction is executed, the key risk signal is taken as the risk point, risk point description information is generated based on the description information corresponding to the risk point, and the risk type of the risk point and the location information of the risk point in the input text sequence are determined. Execute the first output instruction to use the risk type, risk point description information, and location information of the risk point as the risk information of the risk point, and store the risk point and the risk information of the risk point together in the risk list.

3. The method according to claim 2, characterized in that, The evidence information corresponding to each risk point, retrieved from a heterogeneous domain knowledge base based on the risk information of each risk point, includes: The semantic similarity between multiple evidence entries stored in the heterogeneous domain knowledge base and the risk description information of the risk point is calculated respectively, and evidence entries with semantic similarity greater than a first preset threshold are selected as first evidence entries, and the score of the first evidence entry is determined according to the semantic similarity corresponding to the first evidence entry. Key features are extracted from the risk description information of the risk point, and the key features are matched with the key features of multiple evidence entries stored in the heterogeneous domain knowledge base to determine the feature matching degree between the risk description information of the risk point and each evidence entry. Evidence entries with a feature matching degree greater than a second preset threshold are selected as second evidence entries, and the score of the second evidence entry is determined according to the feature matching degree corresponding to the second evidence entry. Based on the scores of each first evidence item and each second evidence item, a predetermined number of evidence items are selected from the first and second evidence items as evidence items for the risk point. The evidence source of the evidence entry for the risk point is obtained from the heterogeneous domain knowledge base, and the evidence entry for the risk point and the evidence source of the evidence entry are used as the evidence information corresponding to the risk point.

4. The method according to claim 3, characterized in that, The process of determining the evidence confidence score for each risk point based on the evidence information corresponding to each risk point includes: Each risk point is compared with multiple evidence items corresponding to each risk point to determine the evidence status of each evidence item corresponding to each risk point. The evidence status includes confirmed status, conflicting status, or questionable status. Based on the evidence status of each evidence item, determine the evidence confidence score corresponding to each risk point.

5. The method according to claim 4, characterized in that, The process of updating the risk information of each risk point in the risk list based on the evidence confidence score corresponding to each risk point, to obtain the updated risk information of each risk point, includes: The evidence confidence score, evidence source, and evidence status of each evidence item corresponding to each risk point are added to the risk information of each risk point to obtain the updated risk information of each risk point.

6. The method according to claim 1, characterized in that, The process of determining the risk score for each risk point based on the updated risk information includes: The large language model uses the second thinking chain prompt template to perform risk assessment on each risk point based on the updated risk information of each risk point, and determines the risk score of each risk point.

7. The method according to claim 6, characterized in that, The second thought chain prompt template records a second instruction sequence, which includes a risk assessment instruction, a classification judgment instruction, an adjustment instruction, and a second output instruction arranged in sequence. The process of using the second thinking chain prompt template to perform risk assessment on each risk point based on the updated risk information of each risk point, and determining the risk score of each risk point, includes: Execute the aforementioned risk assessment instructions to determine the degree of impact of each risk point on the user's credit score; Execute the classification judgment instruction to determine the initial level of each risk point based on the degree of impact of each risk point on the user's credit score; Execute the adjustment instruction to adjust the initial level of each risk point based on the updated risk information of each risk point, thereby obtaining the severity level of each risk point; Execute the second output instruction to determine the risk score of each risk point based on the severity level of each risk point.

8. The method according to claim 1, characterized in that, The decision-making process, based on the risk score of each risk point and the corresponding evidence confidence score, includes: The overall score for each risk point is determined based on the risk score of each risk point, the confidence score of the evidence corresponding to each risk point, the total number of risk points, and the risk score balance adjustment coefficient. The overall score of each risk point is mapped to obtain the rejection probability corresponding to each risk point; The decision result is determined based on the rejection probability corresponding to each risk point, and the decision result includes any one of the following: pass, reject, or review.

9. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when the electronic device is in operation, are executed by the processor to perform the steps of the risk decision-making method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the risk decision-making method as described in any one of claims 1 to 8.

Images