Knowledge memory updating method of AI modeling system

Abstract

The embodiment of the application provides a knowledge memory updating method of an AI modeling system, and relates to the technical field of artificial intelligence. The method comprises the following steps: determining to-be-confirmed knowledge of a modeling object from multiple-source knowledge of an AI modeling system; generating a knowledge object of the to-be-confirmed knowledge based on the to-be-confirmed knowledge; the knowledge object comprises a verification path and verification indexes under the verification path; the verification path comprises one or more of data verification, modeling verification and document verification; verification is performed through any verification path indicated by the knowledge object, and index values corresponding to the verification indexes under the verification path are obtained; a verification result under the verification path is determined according to the index values corresponding to the verification indexes; and it is determined whether the to-be-confirmed knowledge is added to the knowledge memory of the modeling object according to the verification result of the knowledge object under each verification path. Through verification of the to-be-confirmed knowledge instead of directly updating the knowledge memory with the to-be-confirmed knowledge, the problem that model training is wrong due to unreliable knowledge is avoided.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a knowledge memory update method for an AI modeling system. Background Technology

[0002] In the financial technology field, anti-money laundering models are typically developed collaboratively by business personnel, data developers, and modelers. In recent years, with the continuous development of artificial intelligence technology, anti-money laundering models have attempted to incorporate large language models for assisted modeling, such as generating model training schemes or model reports based on natural language requirements.

[0003] Because anti-money laundering AI modeling relies on highly complex knowledge—for example, AI needs to understand whether black samples originate from confirmed suspicious transaction reports, internal cases, or entities identified as high-risk by regulators, and whether white samples need to exclude users with a history of transactions or those with high-risk ratings—and the timeliness of updating this knowledge is also crucial. Therefore, while ensuring rapid updates to the knowledge memory, it is also necessary to verify these updates.

[0004] Currently, knowledge is provided to AI by manually writing prompts, manually organizing knowledge bases, and manually maintaining documents, thereby updating the knowledge memory. However, this cannot guarantee the timeliness of the knowledge memory update, nor can it determine whether the knowledge is accurate or outdated, nor can it feed error logs from the modeling process back into the knowledge memory. Summary of the Invention

[0005] This application provides a knowledge memory update method for an AI modeling system, used to verify the knowledge memory update.

[0006] Firstly, a knowledge memory update method for an AI modeling system includes: The knowledge to be confirmed for the modeling object is determined from the multi-source knowledge of the AI ​​modeling system; Based on the knowledge to be confirmed, a knowledge object of the knowledge to be confirmed is generated; the knowledge object includes a verification path and a verification indicator under the verification path; the verification path includes one or more of data verification, modeling verification and document verification; the data verification characterizes the performance of the knowledge to be confirmed in historical statistical data, the modeling verification characterizes the performance of the knowledge to be confirmed in improving the model results of the modeling object, and the document verification characterizes the performance of the knowledge to be confirmed in documents with different credibility levels. By using any verification path indicated by the knowledge object, the candidate knowledge within the verification scope is verified to obtain the index value corresponding to each verification index under the verification path. The verification results under the verification path are determined based on the index values ​​corresponding to each verification index. Based on the verification results of the knowledge object under each verification path, determine whether the knowledge to be confirmed should be added to the knowledge memory of the modeling object.

[0007] In this embodiment, the knowledge to be verified for the modeling object is determined from the multi-source knowledge of the AI ​​modeling system, rather than directly updating the knowledge memory with multi-source knowledge. This avoids model training errors caused by unreliable knowledge in the knowledge memory. Through multiple methods, such as data verification, model verification, and document verification, the knowledge to be verified is validated in terms of its authenticity, reliability, and usability, thereby improving the accuracy of the knowledge to be verified and ultimately enhancing the accuracy of model training.

[0008] Optionally, based on the knowledge to be confirmed, a knowledge object of the knowledge to be confirmed is generated, including: Based on the knowledge attributes of the knowledge to be verified, the verification path of the knowledge to be verified and the verification index under each verification path are determined; wherein, if the knowledge attribute is a candidate feature, the verification path is data verification; if the knowledge attribute is a candidate rule, the verification path is modeling verification; if the knowledge attribute is candidate knowledge, the verification path is document verification.

[0009] In this embodiment of the application, by generating a knowledge object from the knowledge to be verified, and determining the verification path and the verification indicators of each verification path based on the knowledge attributes of the knowledge object, the accuracy of the verification of the knowledge to be verified is improved, the knowledge to be verified can be used more reliably, and the verification of the knowledge to be verified is avoided due to errors in the verification path of the knowledge to be verified, thereby improving the reliability of the verification of the knowledge to be verified.

[0010] Optionally, the knowledge object further includes a verification scope, which is composed of candidate knowledge expanded based on the knowledge to be confirmed; The verification path is data verification, and the verification scope is to instantiate and expand the data scope in the knowledge to be confirmed. The verification path is modeling verification, and the verification scope is instantiating the rule range in the knowledge to be confirmed.

[0011] In this embodiment of the application, by expanding the data range of the knowledge to be confirmed under data verification and expanding the rule range of the knowledge to be confirmed under modeling verification, the knowledge to be confirmed is made more realistic, and the accuracy of the verification of the knowledge to be confirmed is improved.

[0012] Optionally, verification is performed through any verification path indicated by the knowledge object to obtain the indicator values ​​corresponding to each verification indicator under the verification path, including: Verify any candidate knowledge within the verification scope to obtain the index values ​​corresponding to each verification index of the candidate knowledge under the verification path; Based on the index values ​​corresponding to each verification index, the verification results under the verification path are determined, including: Based on the index values ​​corresponding to each verification indicator for each candidate knowledge within the verification scope, the verification result of each candidate knowledge is obtained, and the verification result of the knowledge to be confirmed under data verification is determined based on the verification results of each candidate knowledge.

[0013] In this embodiment of the application, by verifying each candidate knowledge, the verification results of each candidate knowledge are obtained, and then the verification results of the knowledge to be confirmed are determined by the verification results of each candidate knowledge, so that the verification results of the knowledge to be confirmed are more comprehensive and more accurate.

[0014] Optionally, based on the index values ​​corresponding to each verification index for each candidate knowledge within the verification scope, the verification result for each candidate knowledge is obtained, including: For the same verification metric, the metric value corresponding to each candidate knowledge is normalized. The weight of each verification indicator is determined based on its discriminatory power over the verification range. The verification result of each candidate knowledge is determined based on the weight of each verification indicator and the normalized indicator value of each candidate knowledge under each verification indicator.

[0015] In this embodiment, by normalizing each verification index and then determining the weight of each verification index, the verification result of each candidate knowledge is determined, making the verification result of each candidate knowledge more accurate.

[0016] Optionally, the verification path is data verification, and the verification result of each candidate knowledge is determined based on the indicator value of each candidate knowledge in each verification indicator and the indicator value abnormality penalty item. The verification path is modeling verification. The verification result of each candidate knowledge is determined based on the index value, overfitting penalty term, stability penalty term, and false alarm penalty term corresponding to each verification index.

[0017] In this embodiment of the application, a penalty term is added to the verification results of data verification and modeling verification to make the verification results more accurate.

[0018] Optionally, determining the knowledge to be confirmed about the modeling object from the multi-source knowledge of the AI ​​modeling system includes: Extract the modeling information associated with the modeling object from the technical documentation; Determine whether the modeling information is an outdated signal during the operation of the technical document and / or the modeling object; if so, treat the modeling information as knowledge to be confirmed. Determine whether the modeling information differs from the existing knowledge memory of the modeling object; if so, treat the modeling information as knowledge to be confirmed. Determine whether the modeling information differs from the running result of the modeling object; if so, treat the modeling information as knowledge to be confirmed.

[0019] In this embodiment of the application, modeling information is extracted from technical documents and compared with outdated signals during the modeling object's operation, with the modeling object's operation results, and with the modeling object's existing knowledge. If there are differences, the modeling information is regarded as knowledge to be confirmed, making the knowledge to be confirmed more reliable and more accurate.

[0020] Optionally, determining the knowledge to be confirmed about the modeling object from the multi-source knowledge of the AI ​​modeling system includes: Based on blacklisted objects, candidate risk objects are identified from object profile information and transaction flow information; The candidate risk objects are used as the knowledge to be confirmed.

[0021] In this embodiment of the application, candidate risk objects are determined based on blacklist objects and then used as knowledge to be confirmed, instead of directly treating risk objects as blacklist objects, thereby improving the accuracy of determining the blacklist.

[0022] Optionally, the step of determining candidate risk targets based on blacklisted targets, using target profile information and transaction flow information, includes: Generate image vectors for blacklisted objects and candidate objects respectively; For any candidate object, the candidate risk object is determined based on the high-risk profile features of the candidate object, the similarity between the profile vector of the blacklist object and the profile vector of the candidate object, and the association strength between the blacklist object and the candidate object.

[0023] In this embodiment, candidate risk objects are determined based on the high-risk profile features of the candidate objects, their similarity to blacklisted objects, and the strength of their association with blacklisted objects, thereby improving the accuracy of candidate risk object confirmation.

[0024] Optionally, the high-risk profile features of the candidate objects are determined by the following methods: Based on the feature vector of the blacklisted object, a profile bin is determined, and the profile bin is one or more risk profile features; For any given image bin, the proportion of blacklisted objects falling into that image bin and normal objects falling into that image bin is used to determine high-risk image characteristics; Determine the high-risk profile features corresponding to the profile vector of the candidate object.

[0025] In this embodiment of the application, the proportion of blacklisted individuals falling into the portrait bin and the proportion of normal individuals falling into the portrait bin can more accurately determine the characteristics of high-risk portraits.

[0026] Optionally, the association strength between the blacklisted objects and the candidate objects is determined by the following methods: A counterparty graph is constructed based on transaction flow. The nodes of the counterparty graph include blacklisted objects, normal objects, and candidate objects. The edges of the counterparty graph are transaction weights. Each transaction weight is determined based on one or more of the following: transaction amount, number of transactions, transaction direction, fund stay time, and time decay factor. For any candidate object, the propagation result of the candidate object to the blacklist objects in the counterparty graph is used as the association strength between the blacklist objects and the candidate object.

[0027] In this embodiment of the application, by constructing a counterparty graph, the correlation strength between candidate objects and blacklist objects can be determined, thereby making the determination of candidate risk objects more accurate based on the correlation strength.

[0028] Secondly, embodiments of this application provide a knowledge memory updating device for an AI modeling system, comprising: The determination module is used to determine the knowledge to be confirmed about the modeling object from the multi-source knowledge of the AI ​​modeling system. The generation module is used to generate a knowledge object of the knowledge to be confirmed based on the knowledge to be confirmed. The knowledge object includes a verification path and verification indicators under the verification path. The verification path includes one or more of data verification, modeling verification and document verification. The data verification characterizes the performance of the knowledge to be confirmed in historical statistical data. The modeling verification characterizes the performance of the knowledge to be confirmed in improving the model results of the modeling object. The document verification characterizes the performance of the knowledge to be confirmed in documents with different credibility levels. The verification module is used to verify candidate knowledge within the verification scope through any verification path indicated by the knowledge object, and obtain the indicator values ​​corresponding to each verification indicator under the verification path; and determine the verification result under the verification path based on the indicator values ​​corresponding to each verification indicator. The determining module is further configured to determine whether the knowledge to be confirmed should be added to the knowledge memory of the modeling object based on the verification results of the knowledge object under each verification path.

[0029] The generation module is specifically used for: Based on the knowledge attributes of the knowledge to be verified, the verification path of the knowledge to be verified and the verification index under each verification path are determined; wherein, if the knowledge attribute is a candidate feature, the verification path is data verification; if the knowledge attribute is a candidate rule, the verification path is modeling verification; if the knowledge attribute is candidate knowledge, the verification path is document verification.

[0030] Optionally, the knowledge object further includes a verification scope, which is composed of candidate knowledge expanded based on the knowledge to be confirmed; The verification path is data verification, and the verification scope is to instantiate and expand the data scope in the knowledge to be confirmed. The verification path is modeling verification, and the verification scope is instantiating the rule range in the knowledge to be confirmed.

[0031] Optionally, the verification module is specifically used for: Data verification is performed on any candidate knowledge within the verification scope to obtain the indicator values ​​corresponding to each verification indicator of the candidate knowledge under the verification path. Based on the index values ​​corresponding to each verification index, the verification results under the verification path are determined, including: Based on the index values ​​corresponding to each verification indicator for each candidate knowledge within the verification scope, the verification result of each candidate knowledge is obtained, and the verification result of the knowledge to be confirmed under data verification is determined based on the verification results of each candidate knowledge.

[0032] Optionally, the verification module is specifically used for: For the same verification metric, the metric value corresponding to each candidate knowledge is normalized. The weight of each verification indicator is determined based on its discriminatory power over the verification range. The verification result of each candidate knowledge is determined based on the weight of each verification indicator and the normalized indicator value of each candidate knowledge under each verification indicator.

[0033] Optionally, the verification module is specifically used for: The verification path is data verification. The verification result of each candidate knowledge is determined based on the indicator value of each candidate knowledge in each verification indicator and the indicator value abnormality penalty item. The verification path is modeling verification. The verification result of each candidate knowledge is determined based on the index value, overfitting penalty term, stability penalty term, and false alarm penalty term corresponding to each verification index.

[0034] Optionally, the verification module is specifically used for: The knowledge to be confirmed is compared with the authoritative documents that are already in effect. The authoritative documents that are not outdated or conflicting are authoritative documents. If the knowledge to be confirmed is not different from the document that has already taken effect, then the effective time of the document that has already taken effect will be updated to the generation time of the knowledge to be confirmed. If the knowledge to be confirmed differs from the effective document, a first knowledge to be confirmed is generated based on the difference result. Data verification and / or modeling verification are then performed based on the first knowledge to be confirmed, thereby obtaining the corresponding indicator value under the data verification and / or modeling verification.

[0035] Optionally, the determining module is specifically used for: Based on blacklisted objects, candidate risk objects are identified from object profile information and transaction flow information; The candidate risk objects are used as the knowledge to be confirmed.

[0036] Optionally, the determining module is specifically used for: Generate image vectors for blacklisted objects and candidate objects respectively; For any candidate object, the candidate risk object is determined based on the high-risk profile features of the candidate object, the similarity between the profile vector of the blacklist object and the profile vector of the candidate object, and the association strength between the blacklist object and the candidate object.

[0037] Optionally, the determining module is specifically used for: Based on the feature vector of the blacklisted object, a profile bin is determined, and the profile bin is one or more risk profile features; For any given image bin, the proportion of blacklisted objects falling into that image bin and normal objects falling into that image bin is used to determine high-risk image characteristics; Determine the high-risk profile features corresponding to the profile vector of the candidate object.

[0038] Optionally, the determining module is specifically used for: A counterparty graph is constructed based on transaction flow. The nodes of the counterparty graph include blacklisted objects, normal objects, and candidate objects. The edges of the counterparty graph are transaction weights. Each transaction weight is determined based on one or more of the following: transaction amount, number of transactions, transaction direction, fund stay time, and time decay factor. For any candidate object, the propagation result of the candidate object to the blacklist objects in the counterparty graph is used as the association strength between the blacklist objects and the candidate object.

[0039] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0040] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of any of the methods described above.

[0041] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program stored on a computer-readable storage medium, the computer program including program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described method. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A system architecture diagram provided for an embodiment of this application; Figure 2 A flowchart illustrating a knowledge memory update method for an AI modeling system provided in this application embodiment; Figure 3 A flowchart illustrating the verification result for determining candidate knowledge, provided as an embodiment of this application; Figure 4 A flowchart illustrating a document verification method provided in an embodiment of this application; Figure 5 A schematic diagram illustrating a process for determining high-risk profile features, provided as an embodiment of this application; Figure 6 A schematic diagram of the structure of a knowledge memory update device for an AI modeling system provided in this application embodiment; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and beneficial effects of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] To facilitate understanding, the application background of the embodiments of this application is introduced below.

[0046] In the financial transaction field, especially in anti-money laundering model development, it is typically a collaborative effort involving business personnel, data developers, and modelers. Business personnel provide anti-money laundering guidelines and regulatory requirements; data developers process samples and features based on data dictionaries and historical SQL statements; and modelers then train the model based on the samples and features, ultimately generating model reports and deployment scripts. In recent years, some systems have attempted to introduce large language models or AI agents to assist in modeling, for example, generating sample SQL statements, feature SQL statements, model training schemes, or model reports based on natural language requirements.

[0047] However, the knowledge relied upon for anti-money laundering AI modeling is extremely complex. Taking a suspicious transaction identification model for private clients as an example, the AI ​​needs to simultaneously understand whether black samples come from confirmed historical suspicious transaction reports (STRs), internal cases, or entities confirmed to be at risk by regulators; whether white samples need to exclude historical STR clients, clients with confirmed cases, clients on valid lists, and clients with high-risk ratings; whether transaction statistical features must use the case date, rating date, or sampling date as the observation cutoff date; whether list hit features need to filter list status, list validity period, risk level, and hit date; whether incoming, outgoing, cash, non-cash, cross-border, and non-face-to-face transactions need to be statistically analyzed separately; and whether the deployed SQL needs to be consistent with the binning boundaries, missing value handling, feature windows, and scoring formulas used in the training phase.

[0048] Current methods typically provide AI with background knowledge through manually written prompts, manually compiled knowledge bases, standard RAG searches, or manually maintained documents. While these methods can supplement the context to some extent, they cannot effectively determine whether the knowledge is accurate, outdated, or applicable to the current modeling task. Nor can they continuously feed error logs, model metrics, expert feedback, and regulatory feedback from the modeling process back into the knowledge memory.

[0049] For example, a certain official document might identify 00:00 to 04:00 as the peak period for money laundering, while business experts might believe that 02:00 to 06:00 better reflects the performance of the organization's clients. Current methods typically require manual selection of one of these criteria, or inputting both criteria into the prompt, making it difficult for AI to determine which criterion is more suitable for the current data. Another example is that historical code might use the `risk_level` field to represent a client's risk level, but this field could become the current status field in the latest data. If AI continues to use this field to process historical features, it could lead to future information leaks.

[0050] Currently, the following problems exist in AI modeling for anti-money laundering: (1) The knowledge sources for anti-money laundering AI modeling are scattered, making it difficult for AI to be used stably and accurately. Anti-money laundering modeling knowledge is scattered in business requirement documents, data dictionaries, historical SQL, historical code, model reports, regulatory documents, industry documents, expert feedback, approval opinions, execution logs, and error logs. Ordinary retrieval can only recall text and cannot determine whether the knowledge is true, applicable, or has been replaced by new standards.

[0051] (2) When knowledge from different sources is inconsistent, there is a lack of an operational verification mechanism. For example, when official documents, business experience and historical codes give different descriptions of high-risk transaction periods at night, customer risk level fields, list hit criteria or white sample exclusion rules, existing solutions often rely on human experience to make decisions and cannot automatically determine which knowledge can be verified by data and which knowledge can only be temporarily effective based on the authority of the source.

[0052] (3) Lack of cross-validation between document description and latest data performance. A field may be described in the document as a customer risk level field, but in reality, it may have an abnormal null value rate, changes in enumerated values, inconsistencies with authoritative rating snapshots, or abrupt changes in distribution in the cross-period validation set (OOT) samples. If AI directly uses this knowledge, it may generate incorrect features or incorrect model conclusions.

[0053] (4) New experiences generated during the modeling process are difficult to retain. Information such as SQL execution failure, sample size expansion after JOIN, white sample contamination, time travel, leakage of future information of list status, abnormally high model AUC, unstable OOT, PSI exceeding the threshold, and expert correction opinions are usually limited to a single task, and subsequent AI modeling tasks may still repeat the same mistakes.

[0054] (5) The use of blacklists and regulatory feedback is too crude. Existing systems mostly use blacklists as hit rules, and rarely use blacklist subject profiles, capital flow profiles, and counterparty network profiles to diffuse and screen candidate objects, and then verify them through data quality, false alarm scenarios, rapid modeling and expert confirmation before condensing them into modeling knowledge.

[0055] See Figure 1 This is a system architecture diagram provided in an embodiment of the present application. The system architecture includes a terminal device 101 and a server 102.

[0056] Terminal device 101 is pre-installed with business applications for AI modeling, which may include client applications, web applications, mini-program applications, etc. Terminal device 101 may be a smartphone, POS machine, desktop laptop, computer, etc., but is not limited to these.

[0057] Server 102 is the backend server for business applications. Server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0058] See Figure 2 The knowledge memory update method for an AI modeling system provided in this application includes the following steps: Step 201: Determine the knowledge to be confirmed for the modeling object from the multi-source knowledge of the AI ​​modeling system.

[0059] In one alternative implementation, financial transactions typically require verification to identify risk and determine whether the user is a legitimate or risky transaction subject. AI modeling utilizes large language models, machine learning models, automated modeling tools, or intelligent agents to assist in sample construction, feature processing, model training, model validation, report generation, and deployment code generation. In anti-money laundering AI modeling systems, anti-money laundering AI modeling knowledge memory refers to the long-term knowledge set supporting anti-money laundering AI modeling, including business definitions, data fields, development specifications, model experience, regulatory feedback, and historical error experience.

[0060] For example, taking money laundering transactions in the financial transaction field as an example, the modeling object is a suspicious transaction identification model for private clients. The AI ​​modeling system collects relevant multi-source knowledge inputs, including business requirement documents, data dictionaries, customer information tables, transaction logs, list matching tables, historical suspicious transaction reports (STR) customer tables, case tables, historical sample SQL, historical feature SQL, historical codes, model reports, regulatory documents, expert feedback, execution logs, error logs, and, in this embodiment, the knowledge to be confirmed generated based on the self-updating closed loop of technical documents, and candidate risk objects generated by the diffusion screening and verification of blacklist objects. The above sources are not directly regarded as the final credible knowledge, but rather as the input sources of knowledge to be confirmed in subsequent processes.

[0061] Step 202: Based on the knowledge to be confirmed, generate a knowledge object for the knowledge to be confirmed; the knowledge object includes a verification path and verification indicators under the verification path; the verification path includes one or more of data verification, modeling verification and document verification; data verification characterizes the performance of the knowledge to be confirmed in historical statistical data, modeling verification characterizes the performance of the knowledge to be confirmed in improving the model results of the modeling object, and document verification characterizes the performance of the knowledge to be confirmed in documents with different levels of credibility.

[0062] An optional implementation method unifies and structures the STR / case sample definition, white sample exclusion rules, observation window, list hit conditions, customer risk level field definition, transaction direction, fund flow and counterparty network characteristics, false alarm scenarios, regulatory feedback, development specifications, model evaluation experience and historical error experience into the knowledge to be confirmed for the anti-money laundering AI modeling system.

[0063] This application does not directly recognize the content in business documents, data dictionaries, historical SQL, historical code, model reports, regulatory documents, expert feedback, execution logs, or model backtesting results as credible knowledge. Instead, it first extracts knowledge to be verified from these materials. The knowledge to be verified may come from business experience, historical development guidelines, model operation feedback, differences in technical documents, regulatory or expert tips, etc. Its purpose is to serve as the verification object for subsequent verification branches, and it does not directly form credible conclusions in the structuring stage.

[0064] The knowledge to be verified is transformed into a unified structured knowledge object. Each knowledge object includes at least the following fields: knowledge type, source text, target table, target field, candidate scope, applicable routes, validation metrics, source list, status, confidence, and version. The candidate scope describes the comparable candidate range, the applicable routes record the paths for subsequent data validation, modeling validation, or authoritative document validation, the source list only records the knowledge source and version, and does not directly indicate that the knowledge has been verified as reliable, and the status is "pending verification" by default at this stage.

[0065] Data validation characterizes the performance of the knowledge to be validated within historical statistical data, addressing the question of whether the knowledge can be supported by the historical statistical data itself. If the knowledge to be validated is reliable under data validation, it indicates that the knowledge has high discriminative power. Data validation is used to determine whether the knowledge to be validated is supported by sample statistics, field quality, cross-table consistency, and distribution performance.

[0066] Modeling validation characterizes the performance of the knowledge to be validated in improving the model results of the modeled object, addressing the question of whether the knowledge to be validated is truly effective after being added to the model. If the knowledge to be validated is deemed credible under modeling validation, then the knowledge to be validated indicates that it can improve the model's training. Modeling validation is used to determine whether the knowledge to be validated, after being added to the modeling process, can improve the model's performance and maintain stability over time.

[0067] Document validation characterizes the performance of knowledge to be verified across documents of varying credibility. It is used to score or vote on data from sources such as regulatory documents, internal policies, approval records, business experts, data dictionaries, historical code, and model reports in scenarios with insufficient data, insufficient samples, or strong regulatory constraints.

[0068] For example, for risk-period-related knowledge such as "02:00 to 06:00 may be an active window for money laundering," the knowledge to be confirmed can be formed into a structured knowledge object with a general field structure, as shown in Table 1. Table 1

[0069] The field structure in the above example is not limited to risk period knowledge; it can also be reused for different types of knowledge to be confirmed, such as field definitions, sample rules, list hit conditions, observation windows, feature processing logic, and fund flow characteristics. Instead of directly outputting whether the knowledge to be confirmed is credible based on the structured results, the process continues with a verifiability determination and verification branch selection process.

[0070] Step 203: Obtain the indicator values ​​corresponding to each verification indicator under any verification path indicated by the knowledge object; determine the verification results under the verification path based on the indicator values ​​corresponding to each verification indicator.

[0071] In one optional implementation, the knowledge to be verified is verified according to the verification path indicated by the knowledge object, and the index values ​​corresponding to each verification index of the knowledge to be verified under each verification path are obtained, thereby determining the verification result. Any verification path for the knowledge to be verified can be one of data verification, modeling verification, and document verification, or a combination of multiple paths.

[0072] Step 204: Based on the verification results of the knowledge object under each verification path, determine whether the knowledge to be confirmed should be added to the knowledge memory of the modeling object.

[0073] In one optional implementation, the verification result can be data verification, modeling verification, or document verification, or it can be a comprehensive verification result from multiple verification paths. Based on the verification result, the reliability of the knowledge to be verified is determined, thereby determining whether to add it to the knowledge memory of the modeling object.

[0074] After the three verification paths are completed, the AI ​​modeling system does not directly output a conclusion, but instead proceeds to the result merging stage. Let the knowledge to be verified be k. The AI ​​modeling system can obtain the data verification score DataScore(k), the modeling verification score ModelScore(k), and the document verification score AuthorityScore(k), respectively. If a verification branch is not executed, the weight corresponding to that branch is set to 0 and it does not participate in the result merging.

[0075] The AI ​​modeling system uses a weighted average method to form the final credibility score of the knowledge to be confirmed, as shown in formula (1). Based on the credibility score of the knowledge to be confirmed, it is determined whether the knowledge to be confirmed can update the knowledge memory.

[0076] Where λ_D, λ_M, and λ_A represent the combined weights of data validation, modeling validation, and document validation on the knowledge to be confirmed, respectively. Each weight can be determined based on the sufficiency of evidence, data coverage, source reliability, recent validation time, and suitability for the current task. This formula is only used to merge the standardized scores of executed branches and does not introduce additional duplicate deductions.

[0077] For example, when the verification result When the value exceeds a preset threshold, the credibility of the knowledge to be confirmed is considered high, and it is marked as officially effective, meaning the knowledge to be confirmed can be added to the knowledge memory of the modeling object; when the verification result... When the knowledge is in the candidate range, its credibility is generally low. The knowledge to be confirmed is marked as requiring manual verification to determine whether it can be added to the knowledge memory of the modeling object. When the verification result... When the level is below a preset threshold, the credibility of the knowledge to be verified is low, and the knowledge to be verified is marked as invalid, meaning it cannot be added to the knowledge memory of the modeling object. All verification records, merged scores, processing conclusions, and verification results are written into the knowledge memory for subsequent anti-money laundering AI modeling tasks to retrieve and trace.

[0078] For example, for the unconfirmed knowledge that "02:00 to 06:00 is the active time window for money laundering," it is first structured into a risk period knowledge object, generating multiple candidate knowledge items and determining whether they can undergo data verification. If the time window or its derived features need to be included in the model to verify their actual effect, then modeling verification is further performed. If regulatory documents or business experts also provide relevant time period definitions, document verification is also performed simultaneously. Subsequently, the AI ​​modeling system calculates the scores under data verification, modeling verification, and document verification respectively, and merges the results through each score to obtain the verification result. The verification result ultimately determines whether the unconfirmed knowledge that "02:00 to 06:00 is the active time window for money laundering" is officially effective, awaits manual confirmation, or is ineffective.

[0079] In some embodiments, generating a knowledge object based on the knowledge to be verified includes: determining the verification path of the knowledge to be verified and the verification index under each verification path according to the knowledge attributes of the knowledge to be verified; wherein, if the knowledge attribute is a candidate feature, the verification path is data verification; if the knowledge attribute is a candidate rule, the verification path is modeling verification; if the knowledge attribute is candidate knowledge, the verification path is document verification.

[0080] One optional implementation method involves determining the verification path and verification metrics for the knowledge to be verified based on its knowledge attributes. These knowledge attributes include the knowledge type, available data, sample conditions, regulatory constraints, and historical document information.

[0081] If the knowledge attribute for the knowledge to be confirmed, namely "the active money laundering time window from 02:00 to 06:00," is a candidate feature, then the verification path is data verification. If the knowledge attribute for the knowledge to be confirmed, namely "a rapid increase in the number of trading counterparties in the short term may correspond to abnormal fund flows," is a candidate rule, then the verification path is modeling verification. If the data dictionary describes transaction_amount as "transaction amount," and the null value rate of transaction_amount is significantly increased, and the cross-table consistency rate with the clearing amount field and the order amount field is decreased, then the knowledge attribute is candidate knowledge, and the verification path is document verification.

[0082] In some embodiments, the knowledge object further includes a verification scope, which is composed of candidate knowledge expanded based on the knowledge to be confirmed; the verification path is data verification, and the verification scope is the instantiation and expansion of the data scope in the knowledge to be confirmed; the verification path is modeling verification, and the verification scope is the instantiation of the rule scope in the knowledge to be confirmed.

[0083] One optional implementation path is data verification, where the verification scope expands the data range in the knowledge to be confirmed. For example, for the knowledge to be confirmed, "the active money laundering time window from 02:00 to 06:00", candidate knowledge such as 00:00 to 04:00, 01:00 to 05:00, 02:00 to 06:00, and 03:00 to 07:00 can be generated as the verification scope. Another implementation path is modeling verification, where the verification scope expands the rule range in the knowledge to be confirmed. For example, for the knowledge to be confirmed, "a rapid increase in the number of trading counterparties in the short term may correspond to abnormal fund flows", candidate rules such as "the number of newly added trading counterparties in the past 7 days", "the change in the concentration of trading counterparties in the past 7 days", and "the proportion of funds deposited and quickly withdrawn in the past 7 days" can be generated as the verification scope.

[0084] In some embodiments, verification is performed through any verification path indicated by the knowledge object to obtain the indicator values ​​corresponding to each verification indicator under the verification path, including: verifying any candidate knowledge in the verification scope to obtain the indicator values ​​corresponding to each verification indicator of the candidate knowledge under the verification path. Based on the indicator values ​​corresponding to each verification indicator, determine the verification results under the verification path, including: obtaining the verification results of each candidate knowledge based on the indicator values ​​corresponding to each verification indicator in the verification scope, and determining the verification results of the knowledge to be confirmed under data verification based on the verification results of each candidate knowledge.

[0085] In one optional implementation, during data validation, the validation metrics can be: risk sample hit rate, normal sample hit rate, core risk discrimination metrics (Lift, IV), and rule noise rate. During modeling validation, the validation metrics can be: area under the curve (AUC) improvement, maximum separation (KS) improvement, recall rate improvement for specific suspicious types, false positive rate change, population stability (PSI), out-of-time (OOT) stability, feature contribution, group improvement, and training-OOT difference, etc.

[0086] For example, in data validation, the periods 00:00 to 04:00, 01:00 to 05:00, 02:00 to 06:00, and 03:00 to 07:00 represent candidate knowledge points. Each candidate knowledge point corresponds to five validation metrics: risk sample hit rate, normal sample hit rate, core risk differentiation indicators (Lift and IV), and rule noise rate. The metric values ​​for each candidate knowledge point under these five validation metrics are calculated to obtain the validation result for each candidate knowledge point. If there are other modeling or document validation processes, the validation results from these three paths are combined to obtain the validation result for the knowledge to be confirmed.

[0087] In modeling validation, for example, "number of new trading counterparties in the past 7 days," "change in trading counterparty concentration in the past 7 days," and "proportion of funds quickly withdrawn after being credited in the past 7 days" are candidate knowledge items. Each candidate knowledge item corresponds to validation metrics such as area under the curve (AUC) improvement, maximum separation (KS) improvement, recall rate improvement for specific suspicious types, false positive rate change, population stability (PSI), out-of-time (OOT) stability, feature contribution, grouping improvement, and training-OOT difference. The metric values ​​of each candidate knowledge item under the validation metrics are calculated to obtain the validation result of each candidate knowledge item. If there are other data validations or document validations, the validation results of the three validation paths are combined to obtain the validation result of the knowledge item to be confirmed.

[0088] In some embodiments, the verification result of each candidate knowledge is obtained based on the index value corresponding to each verification index for each candidate knowledge in the verification scope, such as... Figure 3 As shown, it includes the following steps: Step 301: For the same verification metric, normalize the metric value corresponding to each candidate knowledge in the verification metric.

[0089] Step 302: Determine the weight of each verification indicator based on its discriminatory power over the verification scope.

[0090] Step 303: Determine the verification result of each candidate knowledge based on the weight of each verification indicator and the normalized indicator value of each candidate knowledge under each verification indicator.

[0091] An optional implementation, for data verification, involves determining a candidate knowledge set, denoted as C(k), where each candidate knowledge c can be a candidate time window, a candidate field, a candidate sample exclusion rule, a candidate list hit condition, a candidate transaction direction caliber, or a candidate feature processing logic.

[0092] For each candidate knowledge c, calculate several data validation metrics to form a metric vector:

[0093] Where d_j(c) represents the j-th data validation metric. Different data validation metrics can be configured for different knowledge types. For example, for risk period knowledge, risk sample hit rate, normal sample hit rate, Lift, IV, and rule noise rate can be used; for field caliber knowledge, field non-empty rate, type consistency rate, enumeration coverage rate, cross-table consistency rate, and distribution stability can be used; for sample rule knowledge, risk sample retention rate, normal sample exclusion rate, sample contamination rate inverse score, and sample size stability can be used; and for list hit knowledge, list hit rate, effective list coverage rate, risk sample hit improvement, and normal sample hit rate inverse score can be used.

[0094] Normalize the data validation metrics: For positive indicators where larger is better, normalization is performed using formula (2):

[0095] For negative indicators where smaller values ​​are better, normalization is performed using formula (3):

[0096] Where min(d_j) and max(d_j) represent the minimum and maximum values ​​of the j-th indicator on the candidate knowledge set C(k), respectively, and ε is a minimal constant to prevent the denominator from being zero. After the above processing, all indicators are uniformly converted into scores between 0 and 1, and the higher the score, the more sufficient the data validation.

[0097] Weight calculation for each verification indicator. After the indicators are normalized, the weights of each verification indicator are determined based on their discriminative power in the candidate knowledge set, as shown in formula (4):

[0098] Where Var(norm_j) represents the variance of the j-th normalized index in the candidate knowledge set. The larger the variance, the better the validation index can distinguish different candidate knowledge, and the greater its contribution to the selection of candidates, thus it is given a higher weight; if a validation index has small differences among all candidate knowledge, it indicates that its discrimination ability is weak, and its weight is reduced accordingly.

[0099] Core risk differentiation indicators. For candidate knowledge related to risk identification, Lift and IV can be further calculated as core data validation indicators. Among them, Lift can be expressed as shown in formula (5):

[0100] Here, HitRate_risk(c) represents the proportion of risky samples that match candidate knowledge c, and HitRate_normal(c) represents the proportion of normal samples that match candidate knowledge c. The higher the Lift, the better the candidate knowledge can distinguish between risky and normal samples.

[0101] IV can be expressed as follows, according to formula (6):

[0102] Where b represents binning or grouping, p_risk,b(c) represents the proportion of risky samples in the b-th group, and p_normal,b(c) represents the proportion of normal samples in the b-th group. The higher the IV, the stronger the information content of the candidate knowledge in terms of sample differentiation.

[0103] In some embodiments, the verification path is data verification, and the verification result of each candidate knowledge is determined based on the indicator value of each candidate knowledge in each verification indicator and the indicator value anomaly penalty item.

[0104] An alternative implementation involves calculating a data validation score under data validation. The data validation score for candidate knowledge c is shown in formula (7):

[0105] Where DataPenalty(c) represents the data anomaly penalty item, including situations such as severe field missing, cross-table inconsistency, insufficient sample size, abnormal expansion of sample size after JOIN, abnormal drift of enumeration values, insufficient coverage of candidate rules or excessive hit rate of normal samples; ρ is the anomaly penalty coefficient.

[0106] The candidate knowledge with the highest data validation score and that meets the minimum data quality requirements is taken as the data validation recommendation result, as shown in formula (8):

[0107] in, This is to validate the data.

[0108] After completing the data validation for each candidate knowledge, the system does not directly decide whether the knowledge to be confirmed should be used to update the knowledge memory of the modeling object during the data validation stage. Instead, it outputs DataScore(k) and the recommended candidate knowledge c_D. Key performance indicators (KPIs) and data quality descriptions. If a candidate knowledge performs well in metrics such as risk sample hit rate, Lift, and IV, and data quality metrics such as field null value rate, cross-table consistency rate, and sample coverage meet the requirements, its DataScore(k) will be increased accordingly. If some metrics are weak, or there are problems such as missing fields, cross-table inconsistencies, insufficient sample size, or excessively high normal sample hit rate, DataScore(k) will be reduced through DataPenalty(c) or related negative metrics. The results will be combined with the validation results from subsequent modeling validation and document validation.

[0109] For example, for the knowledge to be confirmed regarding the "active money laundering time window," candidate knowledge such as 00:00 to 04:00, 01:00 to 05:00, 02:00 to 06:00, and 03:00 to 07:00 can be generated, and the hit rate of risk samples, the hit rate of normal samples, Lift, IV, and rule noise rate can be calculated respectively. If 01:00 to 05:00 is significantly better than the original 02:00 to 06:00 in terms of the hit rate of risk samples, Lift, and IV, while the hit rate of normal samples does not increase significantly, then 01:00 to 05:00 will be used as the recommended result for data validation, and this validation result will be combined with the modeling validation result and the document validation result.

[0110] For modeling validation, the candidate set corresponding to candidate knowledge k is determined as C(k). For each candidate knowledge c, a baseline model M_0 is constructed without using the candidate knowledge, and a validation model M_c is constructed with the candidate knowledge added. Candidate knowledge can be represented as new features, feature combinations, sample exclusion rules, observation window constraints, list hit conditions, funding chain rules, or counterparty network features. The performance of the baseline model and the validation model is evaluated on training samples, validation samples, and OOT samples, respectively, and a modeling validation metric vector is formed.

[0111] Where m_j(c) represents the j-th modeling validation metric, which may include AUC improvement, KS improvement, recall improvement for specific suspicious types, false positive rate change, PSI, OOT stability, feature contribution, group improvement, and training-OOT difference, etc.

[0112] Model gain and risk metric calculation. For modeling validation metrics that are better the larger they are, such as AUC improvement, KS improvement, recall improvement, feature contribution, and grouping improvement, the gain metric can be defined as shown in formula (9):

[0113] For modeling validation metrics that are better the smaller they are, such as increased false positive rate, PSI, training-OOT difference, and model volatility, risk metrics can be defined as shown in formula (10):

[0114] Here, Metric_j(M_c) represents the performance of the validation model on the j-th metric, and Metric_j(M_0) represents the performance of the baseline model on the j-th metric. If a certain risk metric is to be as small as possible, then inverse normalization is used in subsequent normalization.

[0115] The modeling validation index is normalized and its weights are calculated. For the positive gain index, normalization is performed using formula (11):

[0116] For negative risk indicators, normalization is performed using formula (12):

[0117] After normalization, adaptive weights are determined according to the discrimination of each validation index, as shown in formula (13):

[0118] in, The variance of the j-th validation metric, The sum of the variances of all validation metrics.

[0119] In some embodiments, the verification path is modeling verification, and the verification result of each candidate knowledge is determined based on the index value, overfitting penalty term, stability penalty term, and false alarm penalty term corresponding to each verification index for each candidate knowledge.

[0120] An optional implementation involves calculating the modeling validation score. The modeling validation score for candidate knowledge c is shown in formula (14):

[0121] Here, OverfitPenalty(c) represents the overfitting penalty term, used to penalize candidates whose performance on training samples is significantly improved but whose performance on validation or OOT samples is unstable; StabilityPenalty(c) represents the stability penalty term, used to penalize situations such as excessively high PSI, significant drift in feature distribution over time, and unstable model contribution; FalseAlarmPenalty(c) represents the false alarm penalty term, used to penalize situations where the false alarm rate increases significantly after adding candidate knowledge. α, β, and γ are the weight coefficients of the corresponding penalty terms.

[0122] The training-OOT difference is shown in formula (15):

[0123] Where Score_train(c) represents the modeling validation score of candidate c on the training samples, and Score_oot(c) represents the modeling validation score of candidate c on the OOT samples. If Gap(c) exceeds the preset stability threshold, it indicates that the candidate knowledge may only be effective during training and there is a risk of instability or overfitting across time, which will increase OverfitPenalty(c) or StabilityPenalty(c).

[0124] The candidate knowledge with the highest modeling validation score and that meets the minimum stability requirement is taken as the modeling validation recommendation result, as shown in formula (16):

[0125] in, To verify the modeling results.

[0126] Rapid modeling and multidimensional validation serves as a lightweight implementation of modeling verification. For candidate knowledge involving multiple features, rules, or combined dimensions, rapid modeling and multidimensional validation can be employed as a lightweight approach. This process utilizes sampled data, a short observation period, and a simplified feature set. It employs logistic regression, scoring cards, decision trees, lightweight ensemble models, or rule-based scoring models for ad hoc backtesting, and compares the baseline model without using the candidate knowledge with the validation model incorporating it.

[0127] The rapid modeling and multi-dimensional validation process does not directly generate the final anti-money laundering detection model. Instead, it is used to determine whether candidate knowledge has stable modeling value. It compares metrics such as AUC, KS, recall rate for specific suspicious types, false positive rate, OOT stability, PSI, feature contribution, and grouping improvement, and incorporates these results into the calculation of ModelScore(k). If both training and OOT samples show stable improvement, and PSI, false positive rate, and feature contribution do not exhibit anomalies, the modeling validation score of the candidate knowledge is increased. If only training samples show significant improvement but OOT samples are unstable, the modeling validation score is reduced through overfitting or stability penalties. If neither training nor OOT samples bring effective improvement, or if the false positive rate increases significantly and feature contribution is unstable, the modeling validation score of the candidate knowledge is further reduced.

[0128] Output of modeling validation results. After completing the modeling validation of candidate knowledge, instead of directly deciding whether the knowledge to be confirmed should be used to update the knowledge memory of the modeling object during the modeling validation stage, the ModelScore(k) and the recommended candidate knowledge c_M are output. Model performance gains, stability metrics, and risk descriptions. If key metrics such as AUC, KS, and recall consistently improve in both validation and OOT samples, and PSI and false positive rate do not exceed the thresholds, then ModelScore(k) will be increased accordingly. If only training samples show significant improvement, but OOT samples are unstable, PSI increases, or false positive rate rises, then ModelScore(k) will be reduced using OverfitPenalty(c), StabilityPenalty(c), or FalseAlarmPenalty(c). If adding this candidate knowledge does not significantly improve model performance, or if the false positive rate, PSI, and training-OOT difference significantly worsen, then the candidate knowledge's ModelScore(k) will be lower accordingly.

[0129] For example, for the unconfirmed knowledge that "a rapid increase in the number of trading counterparties in the short term may correspond to abnormal fund flows," candidate knowledge such as "number of new trading counterparties in the past 7 days," "changes in trading counterparty concentration in the past 7 days," and "proportion of funds deposited and quickly withdrawn in the past 7 days" can be generated. A baseline model without this set of candidate knowledge and a validation model with this set of candidate knowledge can be constructed. If the validation model improves the AUC, KS, recall rate of specific suspicious types, and OOT stability compared to the baseline model, and the PSI and false positive rate do not increase significantly, then the ModelScore(k) of this candidate knowledge is increased accordingly. If the candidate knowledge is only effective in the training samples but fails in the OOT samples, then the ModelScore(k) is reduced through overfitting penalty or stability penalty. Finally, the modeling validation results are combined with the data validation results and document validation results.

[0130] In some embodiments, the knowledge to be verified about the modeling object is determined from the multi-source knowledge of the AI ​​modeling system, such as... Figure 4 As shown, it includes the following steps: Step 401: Extract modeling information associated with the modeling objects from the technical documents; Step 402: Determine whether the modeling information is an outdated signal in the technical documentation and / or the operation of the modeling object; if so, treat the modeling information as knowledge to be confirmed; determine whether the modeling information differs from the existing knowledge memory of the modeling object; if so, treat the modeling information as knowledge to be confirmed; determine whether the modeling information differs from the operation result of the modeling object; if so, treat the modeling information as knowledge to be confirmed.

[0131] One possible implementation approach is to address the significant time-sensitivity of anti-money laundering AI modeling knowledge. Data table structures, field enumerations, sample definitions, observation windows, list status definitions, historical SQL syntax, model evaluation criteria, and regulatory requirements can all change with business system upgrades, regulatory rule changes, model version iterations, or expert feedback. If outdated documents, SQL statements, or model reports are used for an extended period, subsequent AI modeling tasks are prone to problems such as field misuse, sample contamination, time travel, sample inflation after joins, inconsistent feature definitions, or distorted model conclusions.

[0132] Therefore, based on the aforementioned data verification and modeling verification, this application introduces a self-updating closed-loop mechanism oriented towards technical documents. This mechanism does not treat technical documents as static background information, nor does it simply overwrite old knowledge with the latest documents. Instead, it extracts field descriptions, sample rules, observation windows, feature processing logic, SQL development specifications, and model report conclusions from the technical documents as knowledge to be verified. Then, it continuously verifies this knowledge based on existing knowledge memory, the latest data performance, and modeling execution results, forming a closed loop of "technical document extraction—existing knowledge comparison—execution result verification—knowledge memory update—versioning write-back."

[0133] Technical document knowledge extraction. First, modeling information usable by the modeling object is extracted from technical documents such as data dictionaries, historical sample SQL, historical feature SQL, historical code, model reports, development specifications, and approval records. This includes field meanings, field types, enumeration ranges, sample exclusion rules, observation windows, JOIN relationships, feature processing logic, binning boundaries, missing value handling methods, and deployment consistency requirements. This information is then converted into structured modeling information. The modeling information records fields such as source text, source document, target table, candidate scope, applicable routes, validation metrics, status, confidence, version, and last verified time.

[0134] Knowledge obsolescence risk identification. Continuously monitor whether modeling information shows signs of obsolescence in technical documentation and the operation of modeling objects. Obsolescence signals may include: data dictionary version changes, field additions or deletions, abnormally high field null value rates, shifts in enumeration value distribution, historical SQL execution failures, abnormal changes in sample size after JOIN operations, increased feature PSI, decreased OOT performance, changes in expert review opinions, updates to regulatory feedback, and related terms in existing knowledge memory being marked as replaced, downgraded, or invalid. When the above obsolescence signals occur, the modeling information is treated as knowledge to be verified, rather than continuing to assume its validity.

[0135] Cross-validation is performed based on existing knowledge. The modeling information is compared with the officially effective knowledge, historical failure knowledge, expert confirmation conclusions, regulatory feedback conclusions, and historical error correction experience stored in the existing knowledge memory of the modeling object. If there are differences between the modeling information and the existing knowledge memory, the existing knowledge memory is not directly overwritten. Instead, the source of the difference, the difference fields, the applicable time range, and the affected links are recorded, and the modeling information is used to generate knowledge to be confirmed.

[0136] The system updates itself based on the modeling results. It combines the SQL execution logs, field null value rate, enumeration drift, cross-table consistency rate, sample size after JOIN, sample contamination rate, feature PSI, OOT stability, model metrics, and deployment validation results generated for each AI modeling task to determine if there are discrepancies between the modeling information and the modeling object's execution results. If discrepancies exist, the modeling information is treated as knowledge to be confirmed.

[0137] Closed-loop write-back and version control. Verification results, update reasons, applicable scope, historical versions, replacement relationships, and most recent verification time are written back to the knowledge memory. For replaced old knowledge, it is not directly deleted, but its historical version and applicable time range are retained to avoid losing information during subsequent reviews or interpretations of historical models. When subsequent modeling objects call upon knowledge, the latest knowledge that is officially in effect and applicable to the current data scope, business scenario, and model task is prioritized for recall, thereby enabling technical documentation knowledge to continuously update itself as business and data change.

[0138] Document verification can be calculated using formula (17):

[0139] Where i represents the source that supports or opposes the knowledge k to be confirmed; a_i represents the authority weight of the source; vote_i(k) represents the direction or strength of the source's support for the knowledge k to be confirmed. Regulatory documents, formal internal policies, approved documents, and authoritative data dictionaries can be configured with higher weights; business expert opinions, historical codes, model reports, and temporary explanatory documents can be configured with different weights based on historical accuracy, most recent update time, and scope of application.

[0140] For example: 1) The data dictionary defines `transaction_amount` as "transaction amount," and historical feature SQL has long used this field to calculate customer transaction amounts. 2) In a subsequent AI modeling task, the AI ​​modeling system found that the null value rate of `transaction_amount` had significantly increased, and the cross-table consistency rate with the settlement amount and order amount fields had decreased. 3) The AI ​​modeling system will not directly conclude that the data dictionary is incorrect, nor will it directly overwrite historical knowledge. Instead, it will generate a confirmation question regarding "whether `transaction_amount` is still the current transaction amount field," and perform document verification based on the field null value rate, cross-table consistency rate, feature PSI, model contribution, and authoritative document sources. 4) If the document verification results indicate that `settle_amount` or `actual_amount` is more consistent with the current transaction amount definition, the AI ​​modeling system will write the new field definition into its knowledge memory and mark the old definition as replaced or downgraded, while retaining the applicable version of the old definition in the historical model.

[0141] In some embodiments, determining the knowledge to be confirmed for the modeling object from the multi-source knowledge of the AI ​​modeling system includes: determining candidate risk objects based on blacklist objects from object profile information and transaction flow information; and using the candidate risk objects as knowledge to be confirmed.

[0142] In one optional implementation, candidate risk objects are identified based on the profile information and transaction flow information of the blacklist, and these candidate risk objects are treated as knowledge to be confirmed. The AI ​​modeling system uses blacklist subjects, regulatory shared list subjects, partner shared list subjects, internally confirmed case subjects, or historical STR subjects as blacklist subjects. This application embodiment does not directly use the diffused subjects or rules as official blacklists, official risk labels, or official modeling knowledge, but rather transforms them into knowledge to be confirmed for data verification, modeling verification, and document verification.

[0143] In some embodiments, candidate risk objects are determined based on blacklist objects from object profile information and transaction flow information, including: generating profile vectors for blacklist objects and candidate objects respectively; and for any candidate object, determining the candidate risk object based on the high-risk profile features of the candidate object, the similarity between the profile vectors of blacklist objects and candidate objects, and the correlation strength between blacklist objects and candidate objects.

[0144] In one optional embodiment, candidate risk objects are determined based on the high-risk profile features of the candidate objects, the similarity between the profile vectors of the candidate objects and the blacklist objects, and the correlation strength between the blacklist objects and the candidate objects.

[0145] Determining the image similarity. The AI ​​modeling system calculates the image similarity between candidate object u and blacklist object s. For the standardized image vectors z_u and z_s, cosine similarity can be used, as shown in formula (18):

[0146] The image similarity score of candidate object u relative to blacklist object s can be taken as the maximum similarity with all blacklist objects. or average similarity When ProfileScore(u) is higher than the threshold, the AI ​​modeling system considers the profile of the candidate object to be similar to that of the blacklisted object and uses it as a candidate for diffusion.

[0147] AI modeling systems can combine similarity scores, association strength scores, and high-risk profile feature scores to form candidate object diffusion scores, as shown in formula (19):

[0148] in, To score the similarity, For the correlation strength score, This indicates the score of the candidate object matching the high-risk profile features. , , The weights are assigned to the three types of diffusion evidence. The AI ​​modeling system selects the Top-N candidate objects or those exceeding the threshold based on the SpreadScore(u) as candidate risk objects, and further summarizes them into knowledge to be confirmed. For example, "entities with a two-hop funding path to blacklist entities and a short fund stay time are of higher risk" and "a rapid increase in the number of new trading counterparties in the past 7 days may correspond to abnormal fund flows," etc.

[0149] In some embodiments, high-risk profile features of candidate objects are determined by means of the following methods: Figure 5 As shown, it includes the following steps: Step 501: Determine the profile bins based on the feature vectors of the blacklisted objects. Each profile bin consists of one or more risk profile features. Step 502: For any profile bin, determine the proportion of blacklisted objects falling into the profile bin and normal objects falling into the profile bin to identify high-risk profile characteristics. Step 503: Determine the high-risk profile features corresponding to the profile vectors of the candidate objects.

[0150] In one optional implementation, let the blacklist object set be S={s_1,s_2,...,s_m}, and the candidate object set be U={u_1,u_2,...,u_n}. The AI ​​modeling system constructs a profile vector x_u for each blacklist object and candidate object. The profile dimensions include customer type, age binning, industry category, transaction channel, fund stay time, outflow concentration, inflow source dispersion, number of counterparties, counterparty concentration, proportion of non-face-to-face transactions, proportion of cross-border transactions, first-hop transaction relationship, second-hop transaction relationship, and community relationship, etc. Continuous variables are binned or standardized, and categorical variables are encoded or embedded to obtain a unified standardized profile vector z_u.

[0151] High-risk profile features are determined. The AI ​​modeling system compares the distribution of blacklisted objects with that of normal users to identify high-risk profile features in the blacklist. For the profile bin b generated by the high-risk profile features, the AI ​​modeling system can calculate the high-risk profile features as shown in formula (20):

[0152] Where N_risk(b) represents the number of blacklisted objects falling into bin b, N_risk represents the total number of blacklisted objects, N_normal(b) represents the number of normal users falling into bin b, N_normal represents the total number of normal users, and ε is a tiny constant to prevent the denominator from being zero. If ProfileLift(b) is higher than a preset threshold, it indicates that the profile bin is significantly enriched in blacklisted objects, and the AI ​​modeling system treats it as a high-risk profile feature, rather than directly as a formal risk feature.

[0153] In some embodiments, the association strength between blacklisted objects and candidate objects is determined by the following methods: A counterparty graph is constructed based on transaction flow. The nodes of the counterparty graph include blacklist objects, normal objects, and candidate objects. The edges of the counterparty graph are transaction weights. Each transaction weight is determined based on one or more of the following: transaction amount, number of transactions, transaction direction, fund stay time, and time decay factor. For any candidate object, the propagation result of the candidate object in the counterparty graph to the blacklist objects is used as the association strength between the blacklist objects and the candidate objects.

[0154] In one optional implementation, the AI ​​modeling system constructs a counterparty graph G=(V,E) based on transaction flow, where nodes represent entities, such as blacklisted objects, normal objects, and candidate objects, and edges represent transaction weights. The transaction weight w_uv can be calculated by comprehensively considering transaction amount, number of transactions, transaction direction, fund stay time, and time decay factor, as shown in formula (21):

[0155] Where, amount_uv represents the transaction amount between subjects u and v, count_uv represents the number of transactions, decay(Δt_uv) represents the time decay function between the transaction time and the observation deadline, and direction_weight_uv represents the weight of the transaction direction or fund flow. The AI ​​modeling system first sets the initial risk score of the blacklist to 1, and other subjects to 0, as shown in formula (22):

[0156] Then, one-hop or two-hop diffusion is performed using the neighborhood propagation method, as shown in formula (23):

[0157] Where N(u) represents the set of neighbors of subject u, α represents the retention coefficient for jumping back to the original blacklist object, and t represents the diffusion round. t=1 yields the one-hop neighborhood risk propagation result, and t=2 yields the two-hop neighborhood risk propagation result. The higher p_u^(t), the stronger the association between the candidate object and the blacklist object through transaction paths, fund flows, or common trading counterparties.

[0158] The key to this solution is not to complete the diffusion of the blacklist itself, but to transform the diffusion discovery into knowledge to be confirmed and return it to the aforementioned credibility judgment system. For candidate features obtained by profile difference mining, the system can enter the data verification process of innovation point (2) to calculate the hit rate of risk samples, the hit rate of normal samples, Lift, IV, field quality and distribution stability; for candidate rule combinations obtained by similarity matching or graph neighborhood diffusion, the system can enter the modeling verification process of innovation point (3) to compare the AUC, KS, recall, false positive rate, PSI, OOT stability and feature contribution before and after adding the knowledge; for candidate knowledge from regulatory shared lists or partner shared lists, the system can also combine authoritative documents to verify the score. Finally, the FinalScore(k) of innovation point (1) will uniformly output the status of officially effective, pending manual confirmation or ineffective.

[0159] For example, the system uses a batch of entities on a shared regulatory blacklist as risk seeds. It discovers that some unlabeled entities share two-hop counterparties with blacklisted entities, and funds are rapidly withdrawn shortly after being deposited. Furthermore, these unlabeled entities have high profile similarity and neighborhood diffusion scores. The system does not directly classify these unlabeled entities as blacklisted entities. Instead, it generates pending confirmation knowledge: "Entities with two-hop funding paths to blacklisted entities, short fund stay times, and high counterparty concentration have high risk signals." This knowledge then enters the data verification and modeling verification process. Only when statistical discrimination, model contribution, and stability all meet the requirements will it be written into the knowledge memory through the overall process.

[0160] The knowledge memory update method of the AI ​​modeling system described above will be explained below through specific examples.

[0161] Take the knowledge to be confirmed, k = "02:00 to 06:00 is the active time window for money laundering," as an example. The AI ​​modeling system does not directly aggregate all transaction flows, but first generates the nighttime transaction ratio feature at the customer level, and then calculates Lift, IV, DataScore, and ModelScore by combining each customer's own tags (whether they are money laundering users).

[0162] I. Data Validation.

[0163] 1. Example of customer-level feature generation.

[0164] Taking the candidate time window from 02:00 to 06:00 as an example, the AI ​​modeling system first calculates the customer-level metric night_operation_ratio_2_6(i) for each customer i: night_operation_ratio_2_6(i) = Number of transactions by customer i between 02:00 and 06:00 / Total number of transactions by customer i, as shown in Table 2: Table 2

[0165] All subsequent statistics are based on customer-level metrics and customer tags, rather than directly on total transaction volume.

[0166] 2. Example of substituting candidate set C(k).

[0167] The AI ​​modeling system does not only verify the 02:00 to 06:00 range in the original knowledge, but also instantiates and expands the data range in the knowledge to be confirmed, generating multiple candidate knowledge and generating corresponding customer-level indicators for each candidate knowledge, as shown in Table 3:

[0168] Table 3

[0169] 3. Example of substituting into the Lift formula.

[0170] Taking 02:00 to 06:00 as an example, let the threshold for judging "high nighttime transaction ratio" be night_operation_ratio_2_6 ≥ 0.20. As shown in Table 4, assuming there are 100 money laundering users and 1000 normal users in the sample, of which 30 money laundering users and 50 normal users meet this threshold: Table 4

[0171] Meaning: Customers with a night_operation_ratio_2_6 of at least 20% appear 6 times more often than normal users among money laundering users, indicating that this customer-level characteristic has a significant risk-enhancing effect.

[0172] 4. Example of substituting into the IV formula.

[0173] IV is calculated based on customer-level feature binning and customer tags. Taking night_operation_ratio_2_6 as an example, the customer is divided into three bins, as shown in Table 5: Table 5

[0174] Meaning: In bin B3, which handles nighttime transactions, money laundering users account for 30%, while legitimate users account for only 5%, a significant difference. Therefore, this bin makes a substantial contribution to the Indicator Value (IV). The higher the IV, the stronger the information provided by this customer-level characteristic in distinguishing between money laundering users and legitimate users.

[0175] 5. Data validation index vector V_D(c).

[0176] The AI ​​modeling system can construct a data validation indicator vector for each candidate knowledge. Taking 02:00 to 06:00 as an example, five indicators are selected: hit rate of high-risk customers, hit rate of normal customers, Lift, IV, and distribution stability.

[0177] Where 0.30 represents the hit rate of customers with a nighttime transaction ratio among money laundering users, 0.05 represents the hit rate of normal users, 6.00 represents Lift, 0.66 represents IV, and 0.92 represents the distribution stability score of this feature across different time slices.

[0178] 6. Positive index normalization formula.

[0179] Assume the data metrics for each candidate window are as shown in Table 6: Table 6

[0180] Taking Lift as an example, a larger Lift is better. The minimum Lift value in the candidate knowledge set is 3.14, and the maximum is 6.00. The Lift value from 02:00 to 06:00 is 6.00.

[0181] 7. Normalization formula for negative indices.

[0182] Taking the hit rate of normal customers as an example, the lower the hit rate of normal customers, the better. The minimum value in the candidate knowledge set is 0.050, and the maximum value is 0.120. The hit rate of normal customers from 02:00 to 06:00 is 0.050.

[0183] Meaning: To minimize the occurrence of this metric by normal customers, 02:00 to 06:00 is the optimal value among the candidates in this group.

[0184] 8. Verify the weight w_j of the indicator.

[0185] The AI ​​modeling system first normalizes the validation metrics for each candidate knowledge set, and then calculates the variance of each validation metric within the candidate knowledge set. Assuming the following variances and weights are obtained, as shown in Table 7: Table 7

[0186] Meaning: Lift has a high discriminative power in the current candidate knowledge set, and therefore receives a high weight in DataScore.

[0187] 9. DataPenalty(c) is used to characterize the abnormal risks of candidate knowledge in terms of data quality, sample coverage, and processing methods. It is not a positive indicator to prove the validity of candidate knowledge, but rather to deduct points for unreliable data foundations, preventing indicators such as Lift and IV from being artificially inflated due to missing data, inconsistent methods, or insufficient sample size.

[0188] First, convert different types of data anomalies into penalty scores P_r(c) between 0 and 1, then sum them up by weighted average according to the anomaly type weight η_r:

[0189] Where P_r(c) represents the penalty for the r-th type of data anomaly, η_r represents the weight of this type of anomaly, and usually satisfies Ση_r=1. The larger P_r(c) is, the more severe the data anomaly of this type.

[0190] For abnormal metrics that are "worse the higher they are," such as field missing rate, JOIN inflation factor, and normal customer hit rate, the following normalization penalty can be used:

[0191] For outlier metrics that are "worse the lower they are," such as sample size and cross-table consistency, the following normalization penalty can be used:

[0192] Taking 02:00 to 06:00 as an example, the AI ​​modeling system has generated the customer-level feature `night_operation_ratio_2_6` and calculated the Lift and IV based on customer labels. Assume the data quality check results for this candidate knowledge are as follows: Transaction time field missing rate 8%, allowable threshold 5%, upper limit 20%; Transaction time cross-table consistency rate 92%, minimum requirement 95%, lower limit 80%; Number of money laundering customers meeting the high nighttime transaction ratio threshold 30, minimum requirement 20; Customer number expansion multiple after JOIN 1.08, allowable threshold 1.05, upper limit 1.20; Normal customer hit rate 5%, allowable upper limit 10%.

[0193] The various punishments can then be calculated as follows:

[0194] If the weights for all five types of anomalies are set to 0.20, then:

[0195] Meaning: Although this candidate knowledge performs well in client-level Lift and IV, it has issues such as missing transaction time fields, low consistency in cross-table time metrics, and a slight increase in the number of clients after JOIN. Therefore, the DataPenalty is 0.120. When calculating the DataScore subsequently, this value will be multiplied by the penalty coefficient ρ and then deducted from the positive data score.

[0196] 10. Example of substituting into the DataScore(c) formula.

[0197] After normalizing the data from 02:00 to 06:00, we obtain the following: hit rate for high-risk customers 0.444, hit rate for normal customers 1.000, Lift 1.000, IV 1.000, and distribution stability 1.000. Based on the aforementioned DataPenalty calculation results, DataPenalty(02:00-06:00) = 0.120, and we set the anomaly penalty coefficient ρ = 0.30.

[0198]

[0199] Meaning: The positive score for customer-level data verification from 02:00 to 06:00 is relatively high; after deducting the penalty for minor data anomalies, the DataScore is still 0.869, indicating that the nighttime transaction ratio feature corresponding to this candidate knowledge still has a strong ability to distinguish between money laundering users and normal users at the data level.

[0200] 11. Example of substituting candidate knowledge data to support recommendation results.

[0201] If the AI ​​modeling system calculates all candidate knowledge as shown in Table 9, the following DataScore is obtained: Table 9

[0202] This means that from a data validation perspective, 02:00 to 06:00 is the best-performing time window among the current candidate knowledge set. However, DataScore is only the output of the data validation branch. Whether it is ultimately written into knowledge memory needs to be determined in conjunction with ModelScore and AuthorityScore in the overall process.

[0203] II. Modeling and Validation.

[0204] The modeling validation still uses the example of "02:00 to 06:00 is the active time window for money laundering". The AI ​​modeling system uses night_operation_ratio_2_6 as candidate knowledge for modeling validation and compares the model performance before and after adding this candidate knowledge.

[0205] 1. Construct the modeling verification index vector V_M(c).

[0206]

[0207] Modeling validation metrics are used to observe whether adding candidate knowledge to the model truly improves the recognition effect, and whether it brings stability or false alarm risks.

[0208] 2. Calculation of model gain Gain_j(c).

[0209] Suppose the baseline model M_0 does not use the candidate knowledge night_operation_ratio_2_6, and the validation model M_c includes this candidate knowledge night_operation_ratio_2_6. If the AUC of the baseline model is 0.720 and the AUC of the validation model is 0.751, then:

[0210] Meaning: After adding the candidate knowledge of the proportion of customer-level nighttime transactions from 02:00 to 06:00, the AUC increased by 0.031.

[0211] 3. The formula for the risk indicator Risk_j(c).

[0212] For metrics where smaller is better, such as an increased false positive rate. If the baseline model has a false positive rate of 0.090 and the validation model has a false positive rate of 0.096, then:

[0213] Meaning: Adding this feature increases the false alarm rate by 0.6 percentage points, and it will be treated as a negative risk indicator or a false alarm penalty factor in the future.

[0214] 4. Example of substituting the normalization formula for modeling indicators.

[0215] Assume the model performance of each candidate knowledge is as shown in Table 10: Table 10

[0216] Taking AUC improvement as an example, the larger the AUC improvement, the better. The minimum value in the candidate knowledge set is 0.010, and the maximum value is 0.031. The AUC improvement from 02:00 to 06:00 is 0.031.

[0217] Taking PSI as an example, a lower PSI is better. The minimum value in the candidate set is 0.07, the maximum value is 0.16, and the PSI from 02:00 to 06:00 is 0.09.

[0218] 5. Model the weights q_j.

[0219] The AI ​​modeling system can automatically assign weights based on the discriminative power of each modeling indicator within the candidate knowledge set. Assuming the calculation yields Table 11: Table 11

[0220] Meaning: The inverse scores of PSI stability and false alarm rate differ significantly in the candidate knowledge set, and therefore have high weights.

[0221] 6. Modeling verification result ModelScore(c).

[0222] After normalizing the candidate features from 02:00 to 06:00, we obtained: AUC improved by 1.000, KS improved by 1.000, recall improved by 1.000, PSI stability increased by 0.778, and false positive rate reverse score increased by 1.000.

[0223]

[0224] Assuming overfit penalty = 0.03, stability penalty = 0.03, and false alarm penalty = 0.03; and penalty weights α = 0.40, β = 0.30, and γ = 0.30, then:

[0225] Meaning: From the perspective of modeling and validation, the addition of night_operation_ratio_2_6 to the model brings a significant improvement in performance, and has lower overfitting, stability and false positive penalties, resulting in a higher ModelScore.

[0226] 7. Training - OOT Gap(c).

[0227] Assuming the modeling validation score for candidate knowledge from 02:00 to 06:00 is 0.82 on the training samples and 0.78 on the OOT samples, then:

[0228] If the preset stability threshold is 0.15, then 0.049 is below the threshold, indicating that the knowledge to be verified is relatively stable between training samples and OOT samples. If Score_train=0.82 and Score_oot=0.60, then Gap=0.268, indicating that the knowledge to be verified may only be effective during the training period, and the overfitting penalty or stability penalty needs to be increased.

[0229] 8. Model and validate the recommendation results.

[0230]

[0231] This indicates that in the modeling validation branch, the customer-level nighttime transaction percentage knowledge corresponding to 02:00 to 06:00 performed best. Ultimately, it still needs to be integrated into the overall process and merged with data validation and document validation.

[0232] III. Determination of the spread of the blacklist.

[0233] The formula for blacklist object diffusion focuses on "candidate risk objects," making it unsuitable to directly apply the unconfirmed knowledge of the 02:00 to 06:00 time window. Therefore, this section uses corresponding blacklist objects, candidate objects, and counterparty diagrams as examples. This example only illustrates how the diffusion score is calculated. The entity obtained from diffusion remains a candidate risk entity, not a confirmed blacklist.

[0234] 1. Calculation of high-risk profile features ProfileLift(b).

[0235] Assume there are 1,000 people on the blacklist, of whom 250 fall into the "funds held for less than 1 hour" bin; there are 100,000 normal users, of whom 8,000 fall into this bin.

[0236]

[0237] Meaning: The proportion of users with "funds remaining for less than 1 hour" in the risk profile is about 3.125 times that of normal users, which can be used as a high-risk profile feature.

[0238] 2. Similarity calculation Sim(u,s).

[0239] Assuming the standardized profile vector of candidate object u is z_u=(0.6,0.4,0.8), and the standardized profile vector of a blacklist object s is z_s=(0.2,0.7,0.4), then the cosine similarity is:

[0240] Meaning: Candidate object u is quite similar to blacklist object s in terms of profile.

[0241] 3. Calculate the similarity score ProfileScore(u).

[0242] If the similarity scores of candidate object u with the portraits of the three blacklisted objects are 0.80, 0.72, and 0.65 respectively, then:

[0243] or

[0244] Depending on the configuration, you can choose the maximum similarity or the Top-K average similarity as the similarity score.

[0245] 4. Example of substituting the formula for the transaction edge weight w_uv.

[0246] Assuming the normalized value of the transaction amount between candidate object u and blacklist object s1 is 0.80, the normalized value of the number of transactions is 0.60, the time decay value is 0.90, and the transaction direction weight is 1.20, then:

[0247] Meaning: There is a strong transactional relationship between candidate object u and blacklist object s1.

[0248] 5. Calculation of neighborhood diffusion p_u^(t+1).

[0249] Suppose candidate object u has three neighbors: blacklist s1, blacklist s2, and normal user v. The edge weights are 0.518, 0.160, and 0.123, respectively. The initial risk score of the blacklist is 1, the initial risk score of normal user v is 0, and candidate object u is not a risk seed, therefore p_u^(0) = 0. Let α = 0.15, then the one-hop diffusion is:

[0250] Meaning: Although candidate object u is not on the blacklist, it receives a high graph diffusion score because there are blacklisted objects with strong transaction associations in its one-hop neighborhood.

[0251] 6. Calculation of the association strength score SpreadScore(u).

[0252] Assume that the profile similarity score of candidate object u is ProfileScore(u)=0.76, the association strength score is GraphScore(u)=0.72, and the high-risk profile feature score is DiffProfileScore(u)=0.80; the weights of the three types of evidence are β_1=0.40, β_2=0.40, and β_3=0.20.

[0253]

[0254] If the diffusion candidate threshold is set to 0.70, candidate object u will enter the candidate risk object list. However, it is still not a confirmed blacklist. This result can be further transformed into knowledge to be confirmed, such as "objects with a strong one-hop transaction association with the risk seed and a high profile similarity are of higher risk", and then return to the data verification, modeling verification, and document verification process to determine credibility.

[0255] Based on the same technical concept, embodiments of this application provide a knowledge memory update device for an AI modeling system, such as... Figure 6 As shown, the device 600 includes: The determination module 601 is used to determine the knowledge to be confirmed of the modeling object from the multi-source knowledge of the AI ​​modeling system; The generation module 602 is used to generate a knowledge object of the knowledge to be confirmed based on the knowledge to be confirmed; the knowledge object includes a verification path and a verification indicator under the verification path; the verification path includes one or more of data verification, modeling verification and document verification; the data verification characterizes the performance of the knowledge to be confirmed in historical statistical data, the modeling verification characterizes the performance of the knowledge to be confirmed in improving the model results of the modeling object, and the document verification characterizes the performance of the knowledge to be confirmed in documents with different credibility levels. The verification module 603 is used to verify candidate knowledge within the verification scope through any verification path indicated by the knowledge object, and obtain the indicator values ​​corresponding to each verification indicator under the verification path; and determine the verification result under the verification path based on the indicator values ​​corresponding to each verification indicator. The determining module 601 is further configured to determine whether the knowledge to be confirmed is added to the knowledge memory of the modeling object based on the verification results of the knowledge object under each verification path.

[0256] The generation module 602 is specifically used for: Based on the knowledge attributes of the knowledge to be verified, the verification path of the knowledge to be verified and the verification index under each verification path are determined; wherein, if the knowledge attribute is a candidate feature, the verification path is data verification; if the knowledge attribute is a candidate rule, the verification path is modeling verification; if the knowledge attribute is candidate knowledge, the verification path is document verification.

[0257] Optionally, the knowledge object further includes a verification scope, which is composed of candidate knowledge expanded based on the knowledge to be confirmed; The verification path is data verification, and the verification scope is to instantiate and expand the data scope in the knowledge to be confirmed. The verification path is modeling verification, and the verification scope is instantiating the rule range in the knowledge to be confirmed.

[0258] Optionally, the verification module 603 is specifically used for: Data verification is performed on any candidate knowledge within the verification scope to obtain the indicator values ​​corresponding to each verification indicator of the candidate knowledge under the verification path. Based on the index values ​​corresponding to each verification index, the verification results under the verification path are determined, including: Based on the index values ​​corresponding to each verification indicator for each candidate knowledge within the verification scope, the verification result of each candidate knowledge is obtained, and the verification result of the knowledge to be confirmed under data verification is determined based on the verification results of each candidate knowledge.

[0259] Optionally, the verification module 603 is specifically used for: For the same verification metric, the metric value corresponding to each candidate knowledge is normalized. The weight of each verification indicator is determined based on its discriminatory power over the verification range. The verification result of each candidate knowledge is determined based on the weight of each verification indicator and the normalized indicator value of each candidate knowledge under each verification indicator.

[0260] Optionally, the verification module 603 is specifically used for: The verification path is data verification. The verification result of each candidate knowledge is determined based on the indicator value of each candidate knowledge in each verification indicator and the indicator value abnormality penalty item. The verification path is modeling verification. The verification result of each candidate knowledge is determined based on the index value, overfitting penalty term, stability penalty term, and false alarm penalty term corresponding to each verification index.

[0261] Optionally, the verification module 603 is specifically used for: The knowledge to be confirmed is compared with the authoritative documents that are already in effect. The authoritative documents that are not outdated or conflicting are authoritative documents. If the knowledge to be confirmed is not different from the document that has already taken effect, then the effective time of the document that has already taken effect will be updated to the generation time of the knowledge to be confirmed. If the knowledge to be confirmed differs from the effective document, a first knowledge to be confirmed is generated based on the difference result. Data verification and / or modeling verification are then performed based on the first knowledge to be confirmed, thereby obtaining the corresponding indicator value under the data verification and / or modeling verification.

[0262] Optionally, the determining module 601 is specifically used for: Based on blacklisted objects, candidate risk objects are identified from object profile information and transaction flow information; The candidate risk objects are used as the knowledge to be confirmed.

[0263] Optionally, the determining module 601 is specifically used for: Generate image vectors for blacklisted objects and candidate objects respectively; For any candidate object, the candidate risk object is determined based on the high-risk profile features of the candidate object, the similarity between the profile vector of the blacklist object and the profile vector of the candidate object, and the association strength between the blacklist object and the candidate object.

[0264] Optionally, the determining module 601 is specifically used for: Based on the feature vector of the blacklisted object, a profile bin is determined, and the profile bin is one or more risk profile features; For any given image bin, the proportion of blacklisted objects falling into that image bin and normal objects falling into that image bin is used to determine high-risk image characteristics; Determine the high-risk profile features corresponding to the profile vector of the candidate object.

[0265] Optionally, the determining module 601 is specifically used for: A counterparty graph is constructed based on transaction flow. The nodes of the counterparty graph include blacklisted objects, normal objects, and candidate objects. The edges of the counterparty graph are transaction weights. Each transaction weight is determined based on one or more of the following: transaction amount, number of transactions, transaction direction, fund stay time, and time decay factor. For any candidate object, the propagation result of the candidate object to the blacklist objects in the counterparty graph is used as the association strength between the blacklist objects and the candidate object.

[0266] Based on the same technical concept, embodiments of this application provide a computer device, which may be a terminal or a server, such as... Figure 7 As shown, it includes at least one processor 701 and a memory 702 connected to at least one processor. In this embodiment, the specific connection medium between the processor 701 and the memory 702 is not limited. Figure 7 Taking the connection between processor 701 and memory 702 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0267] In this embodiment of the application, the memory 702 stores instructions that can be executed by at least one processor 701. By executing the instructions stored in the memory 702, at least one processor 701 can perform the steps included in the knowledge memory update method of the AI ​​modeling system described above.

[0268] The processor 701 is the control center of the computer device, capable of connecting various parts of the computer device via various interfaces and lines. It executes instructions stored in the memory 702 and retrieves data stored in the memory 702. Optionally, the processor 701 may include one or more processing units. The processor 701 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 701. In some embodiments, the processor 701 and the memory 702 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0269] The processor 701 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0270] Memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 702 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory 702 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 702 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0271] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device. When the program is run on the computer device, it causes the computer device to perform the steps of the knowledge memory update method of the AI ​​modeling system described above.

[0272] Based on the same inventive concept, this application provides a computer program product, characterized in that the computer program product includes a computer program stored on a computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer device, the computer device performs the steps of the knowledge memory update method of the above-mentioned AI modeling system.

[0273] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0274] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0275] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0276] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0277] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A knowledge memory update method for an AI modeling system, characterized in that, include: The knowledge to be confirmed for the modeling object is determined from the multi-source knowledge of the AI ​​modeling system; Based on the knowledge to be confirmed, a knowledge object of the knowledge to be confirmed is generated; the knowledge object includes a verification path and a verification indicator under the verification path; the verification path includes one or more of data verification, modeling verification and document verification; the data verification characterizes the performance of the knowledge to be confirmed in historical statistical data, the modeling verification characterizes the performance of the knowledge to be confirmed in improving the model results of the modeling object, and the document verification characterizes the performance of the knowledge to be confirmed in documents with different credibility levels. Verification is performed through any verification path indicated by the knowledge object to obtain the indicator values ​​corresponding to each verification indicator under the verification path; the verification result under the verification path is determined based on the indicator values ​​corresponding to each verification indicator. Based on the verification results of the knowledge object under each verification path, determine whether the knowledge to be confirmed should be added to the knowledge memory of the modeling object.

2. The method as described in claim 1, characterized in that, Based on the knowledge to be confirmed, a knowledge object of the knowledge to be confirmed is generated, including: Based on the knowledge attributes of the knowledge to be verified, the verification path of the knowledge to be verified and the verification index under each verification path are determined; wherein, if the knowledge attribute is a candidate feature, the verification path is data verification; if the knowledge attribute is a candidate rule, the verification path is modeling verification; if the knowledge attribute is candidate knowledge, the verification path is document verification.

3. The method as described in claim 1, characterized in that, The knowledge object also includes a verification scope, which is composed of candidate knowledge expanded based on the knowledge to be confirmed; The verification path is data verification, and the verification scope is to instantiate and expand the data scope in the knowledge to be confirmed. The verification path is modeling verification, and the verification scope is instantiating the rule range in the knowledge to be confirmed.

4. The method as described in claim 3, characterized in that, Verification is performed through any verification path indicated by the knowledge object, obtaining the indicator values ​​corresponding to each verification indicator under the verification path, including: Verify any candidate knowledge within the verification scope to obtain the index values ​​corresponding to each verification index of the candidate knowledge under the verification path; Based on the index values ​​corresponding to each verification index, the verification results under the verification path are determined, including: Based on the index values ​​corresponding to each verification indicator for each candidate knowledge within the verification scope, the verification result of each candidate knowledge is obtained, and the verification result of the knowledge to be confirmed under data verification is determined based on the verification results of each candidate knowledge.

5. The method as described in claim 4, characterized in that, Based on the index values ​​corresponding to each verification indicator for each candidate knowledge within the verification scope, the verification result for each candidate knowledge is obtained, including: For the same verification metric, the metric value corresponding to each candidate knowledge is normalized. The weight of each verification indicator is determined based on its discriminatory power over the verification range. The verification result of each candidate knowledge is determined based on the weight of each verification indicator and the normalized indicator value of each candidate knowledge under each verification indicator.

6. The method as described in claim 4, characterized in that, The verification path is data verification. The verification result of each candidate knowledge is determined based on the indicator value of each candidate knowledge in each verification indicator and the indicator value abnormality penalty item. The verification path is modeling verification. The verification result of each candidate knowledge is determined based on the index value, overfitting penalty term, stability penalty term, and false alarm penalty term corresponding to each verification index.

7. The method according to any one of claims 1 to 6, characterized in that, The step of determining the knowledge to be confirmed for the modeling object from the multi-source knowledge of the AI ​​modeling system includes: Extract the modeling information associated with the modeling object from the technical documentation; Determine whether the modeling information is an outdated signal during the operation of the technical document and / or the modeling object; if so, treat the modeling information as knowledge to be confirmed. Determine whether the modeling information differs from the existing knowledge memory of the modeling object; if so, treat the modeling information as knowledge to be confirmed. Determine whether the modeling information differs from the running result of the modeling object; if so, treat the modeling information as knowledge to be confirmed.

8. The method according to any one of claims 1 to 6, characterized in that, The step of determining the knowledge to be confirmed for the modeling object from the multi-source knowledge of the AI ​​modeling system includes: Based on blacklisted objects, candidate risk objects are identified from object profile information and transaction flow information; The candidate risk objects are used as the knowledge to be confirmed.

9. The method as described in claim 8, characterized in that, The process of identifying candidate risk targets based on blacklisted targets, using target profile information and transaction history information, includes: Generate image vectors for blacklisted objects and candidate objects respectively; For any candidate object, the candidate risk object is determined based on the high-risk profile features of the candidate object, the similarity between the profile vector of the blacklist object and the profile vector of the candidate object, and the association strength between the blacklist object and the candidate object.

10. The method as described in claim 9, characterized in that, The high-risk profile features of the candidate objects are determined through the following methods: Based on the feature vector of the blacklisted object, a profile bin is determined, and the profile bin is one or more risk profile features; For any given image bin, the proportion of blacklisted objects falling into that image bin and normal objects falling into that image bin is used to determine high-risk image characteristics; Determine the high-risk profile features corresponding to the profile vector of the candidate object.

11. The method as described in claim 9, characterized in that, The association strength between the blacklisted objects and the candidate objects is determined by the following methods: A counterparty graph is constructed based on transaction flow. The nodes of the counterparty graph include blacklisted objects, normal objects, and candidate objects. The edges of the counterparty graph are transaction weights. Each transaction weight is determined based on one or more of the following: transaction amount, number of transactions, transaction direction, fund stay time, and time decay factor. For any candidate object, the propagation result of the candidate object to the blacklist objects in the counterparty graph is used as the association strength between the blacklist objects and the candidate object.

Images