Mass list screening method and device, storage medium and equipment

Abstract

The invention provides a mass list screening method and device, a storage medium and equipment, and the method comprises the steps: collecting list data of a plurality of data sources, extracting key fields contained in the list data of each category, and according to a matching rule corresponding to a to-be-screened key field, screening the to-be-screened key field, matching the to-be-screened key field with a key field contained in the list data, determining a category corresponding to the list data to which the to-be-screened key field belongs, and determining a backtracking strategy according to the category of the list data to which the to-be-screened key field belongs, and screening the to-be-screened key field from the successfully matched list data according to a backtracking strategy to obtain a screening result. According to the method, the accuracy and pertinence of data can be improved, misjudgment and missed judgment are reduced, the screening efficiency is improved, high flexibility is achieved, the matching rule and the backtracking strategy can be adjusted according to different screening requirements and scenes, and the method adapts to different application scenes.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, storage medium and equipment for screening a massive list. Background Art

[0002] In today's era of information explosion, it is crucial to quickly and accurately obtain the required information. Whether it is internal knowledge management, customer service support, or individual users' needs for professional knowledge, traditional search methods often fail to meet the requirements of efficiency and accuracy. Intelligent question-answering systems have emerged as the times require. They can not only understand the description of questions in natural language, but also extract the most relevant information from massive data through advanced technologies such as deep learning, and provide accurate answers or solutions. This not only improves the user experience, but also significantly improves work efficiency, reduces labor costs, and creates huge value for enterprises and organizations.

[0003] In the financial sector, with the continuous expansion of business scale and the increasing frequency of financial transactions, risk prevention and control has become a crucial link. Among them, list screening, as one of the core means of risk prevention and control, faces huge challenges in massive data processing and accurate risk identification.

[0004] Traditional list screening methods have exposed many limitations when faced with massive list data from different sources and different risk levels. For example, in the data collection and collation stage, they often rely on manual operations or simple script programs, making it difficult to efficiently integrate list data from multiple channels such as regulatory authorities, internal systems, and cooperative institutions. In addition, data format conversion is prone to errors, and data quality is difficult to guarantee. For example, there is a lack of unified and accurate standards for the processing of key information such as names and ID numbers, which is prone to data duplication, errors, or missing data, affecting the accuracy of subsequent screening. It is difficult to meet the needs of the financial sector for accurate and efficient risk prevention and control in the context of massive data. Summary of the invention

[0005] In view of this, the present invention provides a method, device, storage medium and equipment for screening massive lists, which can improve the accuracy and efficiency of massive list screening and meet the needs of the financial field for accurate and efficient risk prevention and control in the context of massive data.

[0006] In a first aspect, an embodiment of the present invention provides a method for screening a massive list, the method comprising:

[0007] Collecting list data from multiple data sources, wherein the list data includes multiple different categories, and extracting key fields included in the list data of each category;

[0008] According to the matching rules corresponding to the key fields to be screened, the key fields to be screened are matched with the key fields included in the list data to determine the category corresponding to the list data to which the key fields to be screened belong;

[0009] A backtracking strategy is determined according to the category of the list data to which the key field to be screened belongs, and the key field to be screened is screened from the successfully matched list data according to the backtracking strategy to obtain a screening result.

[0010] Furthermore, the list data from multiple data sources are collected, and the list data includes multiple different categories. The key fields included in the list data of each category include:

[0011] Collecting list data from multiple data sources, the list data categories include blacklists, warning lists and shared lists of cooperative institutions;

[0012] The formats of the list data of the multiple data sources are standardized to obtain key fields contained in the list data of each category.

[0013] Furthermore, the method further comprises:

[0014] Remove duplicate data from the list data of the multiple data sources according to the key fields;

[0015] Compare and verify the list data after deduplication with the authoritative data source;

[0016] Correct the list data with abnormal verification results.

[0017] Furthermore, the key fields include key fields with unique attributes and key fields with diverse attributes;

[0018] According to the matching rules corresponding to the key fields to be screened, the key fields to be screened are matched with the key fields included in the list data, and the categories corresponding to the list data to which the key fields to be screened belong are determined to include:

[0019] An exact matching rule is adopted for key fields with unique attributes to match the key fields to be screened with the key fields contained in the list data. When the key fields to be screened are completely consistent with the key fields in the list data, the match is determined to be successful, and the category corresponding to the list data to which the key fields to be screened belong is determined based on the matching result;

[0020] Fuzzy matching rules are used for key fields with diverse attributes to match the key fields to be screened with the key fields contained in the list data. When the matching value between the key fields to be screened and the key fields in the list data is greater than the set threshold, the match is determined to be successful, and the category corresponding to the list data to which the key fields to be screened belong is determined based on the matching results.

[0021] Further, a backtracking strategy is determined according to the category of the list data to which the key field to be screened belongs, and the key field to be screened is screened from the successfully matched list data according to the backtracking strategy, including:

[0022] When the list data to which the key field to be screened belongs is blacklist data, screening is performed from all the list data that successfully matches;

[0023] When the list data to which the key field to be screened belongs is the warning list data, according to the risk level of the warning list data, the data backtracking amount corresponding to the risk level is selected from the successfully matched list data for screening;

[0024] When the list data described in the key field to be screened is the shared list data of a cooperative organization, based on the credibility level of the cooperative organization, the data backtracking amount corresponding to the credibility level is selected from the successfully matched list data for screening.

[0025] Furthermore, the method further comprises:

[0026] When the value representing a specific risk in the screening result is greater than a set threshold, a risk warning is triggered and / or the business process is blocked.

[0027] In a second aspect, an embodiment of the present invention provides a massive list screening device, the device comprising:

[0028] A collection module, used to collect list data from multiple data sources, wherein the list data includes multiple different categories, and extract key fields included in the list data of each category;

[0029] A matching module, used for matching the key field to be screened with the key field included in the list data according to the matching rules corresponding to the key field to be screened, and determining the category corresponding to the list data to which the key field to be screened belongs;

[0030] The backtracking module is used to determine a backtracking strategy according to the category of the list data to which the key field to be screened belongs, and screen the key field to be screened from the successfully matched list data according to the backtracking strategy to obtain a screening result.

[0031] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program is configured to execute the massive list screening method described in any one of the first aspects when running.

[0032] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the massive list screening method described in any one of the first aspects.

[0033] The technical solution provided by the present invention, on the one hand, can ensure the comprehensiveness and diversity of the data by collecting list data from multiple data sources, covering more different categories of information, and by extracting and matching the key fields of each category, it helps to improve the accuracy and pertinence of the data and reduce the possibility of misjudgment and missed judgment. On the second hand, the technical solution adopts automated and intelligent means to process and match data, which can significantly improve screening efficiency and reduce the cost and time of manual intervention. On the third hand, the technical solution also has high flexibility and can adjust the matching rules and backtracking strategies according to different screening needs and scenarios to adapt to different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A flowchart of a method for screening a massive list provided in Embodiment 1 of the present invention;

[0036] Figure 2 A flowchart of collecting list data from multiple data sources and extracting key fields contained in the list data of each category provided in the first embodiment of the present invention;

[0037] Figure 3 A flowchart of determining a backtracking strategy according to the category of the list data to which the key field to be screened belongs, and screening the key field to be screened from the successfully matched list data according to the backtracking strategy, provided in the first embodiment of the present invention;

[0038] Figure 4 A schematic diagram of the structure of a device for screening a massive list provided in Embodiment 2 of the present invention;

[0039] Figure 5A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0040] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0041] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Embodiment 1

[0043] See also Figure 1 , Figure 1 It is a flow chart of a method for screening a massive list provided by an embodiment of the present invention, and the method includes the following steps 11-13.

[0044] Step 11: Collect list data from multiple data sources, where the list data includes multiple different categories, and extract key fields included in the list data of each category.

[0045] This step is mainly the basic work of data collection and collation. The financial field involves many sources of information, such as various violation lists provided by regulatory authorities, early warning lists generated by internal risk monitoring systems, and related lists shared with external cooperative institutions, which constitute the list data from multiple data sources. Different categories of list data may serve different risk prevention and control purposes. For example, the blacklist is mainly for serious crimes or high-risk behavior entities, the early warning list focuses on early detection of potential risks, and the list of cooperative institutions pays more attention to risk associations in cooperative businesses. The key fields are information that can identify the key features of the list subject, such as name, ID number, institution name, etc. Extracting these key fields will help to carry out accurate and efficient matching and screening work in the future, simplifying the complex list data into an operational key information set, and laying a data foundation for the entire screening process.

[0046] Specifically, the list data is collected from different data sources (such as internal databases of financial institutions, data push from regulatory agencies, third-party credit data, etc.). These data formats may vary, including structured tables, semi-structured JSON files, and even some unstructured text descriptions (such as mentions of people at risk in news reports). Afterwards, the data fusion neural network model can be used to process the list data from multiple data sources. The neural network automatically converts the data into a unified vector representation rich in semantic information by learning the data features and patterns of different data sources. For example, for names, it can not only recognize the standard name format, but also handle variants such as nicknames and pen names, and map them to the corresponding semantic space, thereby extracting the key fields contained in the list data of each category. Alternatively, other methods can be used to extract the key fields contained in the list data of each category. This is only used for illustration here and is not used to limit the present invention.

[0047] Step 12: According to the matching rules corresponding to the key fields to be screened, the key fields to be screened are matched with the key fields included in the list data to determine the category corresponding to the list data to which the key fields to be screened belong.

[0048] In this step, we must first formulate corresponding matching rules based on different key field types. For key fields with uniqueness (such as ID card numbers), the matching rules are usually strict and exact matches; for key fields such as names that may have certain variability, there will be exact matches (considering the format, full and half-width, character order, etc. to be exactly the same) and fuzzy matches (such as allowing homophone replacement, reversing the order of names, etc. to be considered as matches under certain conditions) rules. By comparing the key fields of the objects to be screened with the key fields in the collected and sorted list data according to these rules, it can be determined which type of list data the key fields to be screened best match, thereby clarifying their category. This step is like a classification filter, which accurately divides the information to be screened into the corresponding risk category framework so that targeted follow-up measures can be taken.

[0049] For example: Using semantic understanding technology in natural language processing, a fuzzy matching engine is built. The engine no longer performs fuzzy matching based solely on similarity at the character level, but instead deeply understands the semantic meaning of key information such as names and organization names. For example, for name matching, it can identify the possible semantic relationship between "Zhang San" and "Zhang Xiaoshan" or different variants of the same person. By building a semantic knowledge graph, the name is associated with relevant family relationships, regional cultural background and other information, so as to more accurately judge the rationality of the fuzzy match.

[0050] Alternatively, an attention mechanism can be used to enable the matching engine to focus on key semantic information. For example, when matching company names, the focus is on the company's core business keywords, brand names and other highly recognizable parts, while some common suffixes (such as "Limited Company" and "Group Company") are given lower attention weights to improve the accuracy and efficiency of fuzzy matching.

[0051] Alternatively, collect and analyze customer behavior patterns in historical list data, such as transaction time patterns, transaction location distribution, transaction amount frequency, etc. Build dynamic matching rules based on these behavior patterns, and for customers with abnormal behavior patterns, increase their matching weight or trigger a more in-depth secondary screening even if the matching of names or other key information is at a fuzzy boundary. For example, a customer usually conducts small transactions in China, but suddenly a large overseas transaction occurs during non-normal business hours. At this time, even if the customer's name does not match the list, it will be included in the high-risk watch list for further review based on the abnormal behavior pattern.

[0052] Step 13: Determine a backtracking strategy according to the category of the list data to which the key field to be screened belongs, and screen the key field to be screened from the successfully matched list data according to the backtracking strategy to obtain a screening result.

[0053] Different types of list data require different backtracking strategies due to differences in risk levels and business impacts. For high-risk blacklists, it is often required to conduct a full backtracking of all historical business records, and the backtracking frequency is high, such as a comprehensive screening every day to ensure that no possible risk association is missed. For medium- and low-risk warning lists or lists shared by cooperative institutions, the time range (such as the past six months or one year) and frequency (such as regular screening in weeks or months) of backtracking will be determined based on business characteristics and risk assessment. The backtracking process will also combine matching rules of different strictness (different emphasis on exact matching and fuzzy matching). According to the determined backtracking strategy, the key fields to be screened are screened within the range of the successfully matched list data, and the screening results are finally obtained. The results can indicate whether there is a risk and the approximate degree of the risk of the object to be screened, providing key basis for the risk decision-making of financial institutions (such as whether to approve loans, whether to impose transaction restrictions, etc.), and realizing effective prevention and control and management of financial business risks.

[0054] The implementation method of each step in the embodiment of the present invention is described in detail below.

[0055] In another embodiment of the present invention, see Figure 2, Step 11 collects list data from multiple data sources. The list data contains multiple different categories. Extracting the key fields contained in the list data of each category can be achieved through the following steps 111 - 115.

[0056] Step 111: Collect list data from multiple data sources. The categories of the list data include blacklists, warning lists, and shared lists of cooperative institutions.

[0057] This is mainly a process of collecting list data of multiple categories from different channels. Among them, the blacklist data comes from the list of high - risk illegal and违规主体名单以及金融机构内部确认的严重违规客户等相关记录;预警名单数据涵盖内部风险监测系统依据交易行为、信用变化等因素生成的潜在风险提示名单,还有外部合作机构提供的特定业务风险预警相关信息;合作机构的共享名单则是按照与各合作方签订的协议,接收其分享过来的涉及合作业务风险管控的相关主体名单。通过对这些不同类别名单数据来源的梳理与获取,为后续整体的名单筛查流程奠定基础。

[0058] Step 112: Standardize the format of the list data from the multiple data sources to obtain the key fields contained in the list data of each category.

[0059] In this step, the list data from different data sources may be presented in various formats, commonly including structured database tables (but the data structures and field definitions of different database systems are different), Excel files, CSV files, and even semi - structured JSON formats, etc. First, it is necessary to identify the format of the collected list data of each category, and then use corresponding technical means to convert it into a unified format that is convenient for subsequent processing, such as uniformly converting it into the form of a structured database table or standard JSON - format data. For example, if a shared list of a cooperative institution in Excel file format is received, which contains information such as customer names, contact information, and credit ratings, the openpyxl library in Python can be used to read the Excel data and organize it into a standard list of Python dictionaries (similar to the structure of JSON format). Each dictionary represents a customer record, and the keys of the dictionary correspond to the corresponding field names (such as {"name": "张三", "phone": "123456789", "credit_level": "A"}), which is convenient for subsequent extraction of key fields and operations.

[0060] It should be noted that there are some unclear parts in the original text, such as "高风险违法违规主体名单以及金融机构内部确认的严重违规客户等相关记录" which is not fully translated due to the unclear expression in Chinese. You may need to check and clarify the original text for a more accurate translation.During the format conversion process, we need to pay attention to the inconsistency of field names and data types that represent the same meaning in different data sources, and perform field alignment and standardization operations. For example, some data sources use "customer_name" to represent customer names, while others use "name". They need to be standardized into a standard field name (such as "name"); for data types, such as date fields, some may be strings in the "YYYY-MM-DD" format, and some may be timestamp numeric types. They need to be uniformly converted into a standard date format that meets business processing requirements (such as uniformly converted into the "YYYY-MM-DD" string format) to ensure that data will not have errors due to format issues in subsequent matching, screening and other operations.

[0061] According to the risk prevention and control needs of financial services and the characteristics of various lists, key identification fields are determined as key fields. For blacklists, key fields may include the subject's unique identity (such as personal ID number, enterprise's unified social credit code), subject name (personal name or enterprise full name), violation type, etc.; for early warning lists, key fields may include key indicators related to risk judgment (such as credit score, recent transaction amount change rate, transaction frequency, etc.) and basic information of the subject; for lists shared by cooperative institutions, key fields usually cover key information related to cooperative business (such as the customer's number in the cooperative business, the customer's industry, the name of the cooperative project, etc.).

[0062] Based on the data in the standardized format, these key fields are extracted through programming code or database operation statements. If it is in the database table, use the SELECT statement to select the corresponding fields for extraction (such as SELECT id_number, name, violation_type FROM blacklist_table to extract some key fields of the blacklist); if it is in the converted JSON format data (represented as a dictionary list in Python), you can extract the key fields by traversing the dictionary list and obtaining the value of the corresponding key (such as for record in data_list: key_field_1 =record['id_number']; key_field_2 = record['name'], etc.), and finally obtain the key field information in each category list data for subsequent matching, screening and other processes.

[0063] In some other embodiments of the present invention, after the formats of the list data of the multiple data sources are standardized in step 112 and the key fields included in the list data of each category are obtained, see Figure 2 , the method may further include:

[0064] Step 113: Remove duplicate data from the list data of the multiple data sources according to the key fields.

[0065] After collecting and standardizing the list data from different data sources (such as blacklists, warning lists, lists shared by cooperative institutions, etc.), duplicate records may appear due to partial overlap or input between the various data sources. If these duplicate data are not processed, unnecessary waste of computing resources will be caused in subsequent matching, screening and other processes, and may affect the accuracy of the results (such as repeated statistical risk times, etc.). At this time, based on the key fields extracted previously (key information that can uniquely identify the subject, such as name, ID number, and institution code), each record is compared through a specific data processing algorithm or database operation statement. Once a record is found to be duplicated based on the key field, only one of them will be retained, and the rest of the duplicates will be removed, making the list data more streamlined and accurate, providing a better data foundation for subsequent operations.

[0066] Step 114: Compare and verify the list data after deleting duplicate data with the authoritative data source.

[0067] Although the list data has been collected, standardized, and deduplicated through the previous steps, the accuracy of the data still needs to be further verified. Authoritative data sources are usually official registration databases, industry-recognized standard data warehouses, and other information sources with high credibility and guaranteed accuracy. The list data after deduplication is compared with these authoritative data sources, such as comparing key fields such as name, identity information, and key business attributes of the subject to see if there are any differences, so as to comprehensively check whether the list data conforms to the actual situation and whether there are any errors or outdated information, to ensure the quality of the entire list data is reliable, so that it can truly reflect the risk status of each subject and other related situations, and provide solid guarantees for subsequent accurate screening.

[0068] Step 115: Correct the list data with abnormal verification results.

[0069] After the comparison and verification with the authoritative data source is completed, the corresponding verification results will be obtained. Those list data that are inconsistent with the authoritative data source and do not conform to the actual situation are the data with abnormal verification results. These abnormal data may be caused by various reasons such as data entry errors and untimely information updates. For these abnormal data, further verification and verification (such as manually contacting relevant entities, rechecking original records, etc.) are required, and then they are corrected based on accurate information, so that the list data can be consistent with the authoritative data source or the actual situation in each key field, ensuring the overall accuracy and effectiveness of the list data, so that the subsequent risk screening based on these list data will not be biased or misjudged due to problems with the data itself.

[0070] In some embodiments of the present invention, the key fields include key fields with unique attributes and key fields with diverse attributes. Step 12 matches the key fields to be screened with the key fields contained in the list data according to the matching rules corresponding to the key fields to be screened. Determining the category corresponding to the list data to which the key fields to be screened belong can be achieved by following the steps 121-122.

[0071] Step 121, using an exact matching rule for key fields with unique attributes, matching the key fields to be screened with the key fields included in the list data; when the key fields to be screened are completely consistent with the key fields in the list data, the match is determined to be successful, and the category corresponding to the list data to which the key fields to be screened belong is determined based on the matching result.

[0072] In this step, we first explain the meaning of key fields with unique attributes. In the list data, there are some key fields with unique attributes. Such key fields can usually accurately and uniquely identify a subject. Common examples include ID card number, unified social credit code (for corporate entities), etc. Due to the uniqueness of these key fields, as long as their values ​​are exactly the same, it can be confirmed that the corresponding subject is the same, without ambiguity or ambiguity.

[0073] The exact matching rule requires that the key fields to be screened must be completely consistent with the corresponding key fields in the list data in terms of content, which means that all details, from characters, formats to specific values ​​of each bit, must be exactly the same. For example, when screening whether an individual is on the blacklist, if the ID number is used as the only key field of the attribute, then the ID number of the individual to be screened must correspond to a certain ID number recorded in the blacklist bit by bit, without any difference, including no more or less digits, wrong numbers, or inconsistent formats (such as full-width and half-width problems).

[0074] After comparison according to the above strict exact matching rules, if the keyword fields to be screened exactly match the keyword fields in the list data, it is determined that the matching is successful. Once the matching is successful, the category of the list data to which the successfully matched list data belongs (such as belonging to the blacklist, warning list, or cooperation institution shared list) can be used to determine the category of the list data to which the subject corresponding to the keyword field to be screened belongs. This is like a precise key opening a precise lock. If the key (keyword field to be screened) can exactly fit the lock (keyword field in the list data), then the belonging (category) corresponding to this key can be known, and subsequent corresponding processing measures can be taken based on this category. For example, more strict risk prevention and control measures may need to be taken for the blacklist category, etc.

[0075] Step 122: Adopt fuzzy matching rules for keyword fields with diverse attributes, match the keyword fields to be screened with the keyword fields included in the list data. When the matching value between the keyword fields to be screened and the keyword fields in the list data is greater than the set threshold, it is determined that the matching is successful, and the category of the list data corresponding to the keyword fields to be screened is determined according to the matching result.

[0076] In this step, different from keyword fields with unique attributes, keyword fields with diverse attributes often have multiple reasonable forms of expression but still point to the same subject. For example, a name is a typical example. Because a name may change due to various reasons, such as homophone substitution ("Zhang Feng" and "Zhang Feng"), name order reversal ("Wang Xiaoming" and "Xiaoming Wang"), and the existence of nicknames or aliases. In fact, they probably refer to the same person. In addition, for enterprise names, there may also be diversity situations such as the mixed use of abbreviations and full names, or different spellings due to translation.

[0077] For such keyword fields with diverse attributes, fuzzy matching rules are adopted for processing. Fuzzy matching rules comprehensively consider the similarity degree between keyword fields and use certain algorithms and logics to measure whether they are similar enough to be recognized as a match. This usually involves establishing some rules and models. For example, a homophone library is established for names, and word processing algorithms are used to analyze and judge homophone substitution situations. Multiple factors such as the overall number of text differences and the matching degree of key information are also considered.

[0078] During the fuzzy matching process, the matching value between the key fields to be screened and the key fields in the list data will be calculated. This matching value reflects the quantitative result of the similarity between the two. At the same time, a threshold will be set in advance based on factors such as the actual business situation, past experience, and tolerance for risk. For example, after comprehensive consideration, when the matching value is set to be greater than 0.8 (only an example value, depending on the business), it is determined to be a successful match. In other words, only when the similarity between the two calculated by the algorithm reaches or exceeds this set threshold, they are considered to point to the same subject under the fuzzy matching rules, and the match is determined to be successful.

[0079] When the match is successful according to the fuzzy matching rules, similar to step 121, based on the category of the successfully matched list data, the list data category of the subject corresponding to the key field to be screened can be clearly identified. In this way, even when faced with key fields with multiple reasonable variations, they can be more accurately matched to the appropriate list category, thereby providing an accurate classification basis for subsequent risk screening, backtracking and other operations, ensuring that the entire process can reasonably deal with the different characteristics of various key fields and effectively carry out risk management work.

[0080] In other embodiments of the present invention, see Figure 3 Step 13 determines a backtracking strategy according to the category of the list data to which the key field to be screened belongs. Screening the key field to be screened from the successfully matched list data according to the backtracking strategy can be achieved through the following steps 131-133.

[0081] Step 131: When the list data to which the key field to be screened belongs is blacklist data, screening is performed from all list data that have been successfully matched.

[0082] Blacklist data usually contains information about entities involved in serious illegal and criminal acts, high-risk money laundering activities or other major violations. Once these entities are included in the blacklist, their risk level is extremely high, and any potential association with them may cause huge losses to financial institutions. For example, if an individual or enterprise that is confirmed to be involved in an international money laundering network is not identified and dealt with in a timely manner in financial business, it may cause the financial institution to be involved in illegal transactions and face serious consequences such as huge fines, reputational damage and even legal proceedings.

[0083] When it is determined that the list data to which the key field to be screened belongs is blacklist data, it is necessary to conduct a comprehensive and complete screening of all the list data that have been successfully matched. It is necessary to traverse the entire successfully matched data set and check each record related to the key field of the blacklist one by one. Whether it is transaction details, account association information, or other related business data in historical business records, they must be reviewed in detail. For example, in the bank's transaction system, if it is found that the key field (such as ID number) of an account holder matches the blacklist, then all deposit, withdrawal, transfer, remittance and other transaction records of the account since the account was opened must be carefully checked, and no transaction link that may be risky should be missed, so as to ensure that all potential risks are thoroughly identified and corresponding risk blocking measures are taken in a timely manner, such as freezing accounts, terminating transactions, and reporting to regulatory authorities.

[0084] Step 132: When the list data to which the key field to be screened belongs is warning list data, according to the risk level of the warning list data, select the data backtracking amount corresponding to the risk level from the successfully matched list data for screening.

[0085] The early warning list data covers the information of entities with potential risk signs, but the risk levels of different entities vary. This risk level classification is usually based on a variety of factors, including the trend of changes in the entity's credit score, the abnormality of recent trading behavior, industry risk conditions, and the potential impact of the macroeconomic environment on its business. For example, if an enterprise that originally had good credit has suddenly experienced multiple abnormal transfers of large amounts of funds recently, and most of its counterparties are unknown entities in high-risk regions or industries, its risk level may be assessed as high; while an individual whose credit score has dropped slightly but whose trading behavior is basically stable may be classified as a medium-to-low risk warning category.

[0086] According to the different risk levels of the early warning list data, the corresponding amount of data backtracking is selected from the successfully matched list data for screening, which can effectively monitor risks while reasonably allocating limited computing resources and labor costs. For the entities on the high-risk early warning list, due to the high possibility of risk escalation and the possibility of large losses in the event of a risk event, it may be necessary to backtrack a long period of time (such as one year) and a large proportion (such as more than 80%) of business data for detailed screening. For example, for a trading company marked as a high-risk early warning, it may be necessary to check all its trade contracts, fund transaction records, customs declaration documents and other business materials with domestic and foreign partners in the past year to deeply analyze its potential risk factors and possible risk evolution paths. For the entities on the medium and low risk early warning list, due to their relatively low risks, the backtracking time (such as half a year) and the amount of backtracking data (such as 30%-50%) can be appropriately reduced, and relatively efficient methods such as sampling screening can be used to focus on abnormal changes in recent business data. Trends, so as to promptly discover signs of risk escalation and take corresponding preventive measures, such as strengthening the frequency of transaction monitoring and requiring additional risk guarantees.

[0087] Step 133: When the list data described in the key field to be screened is the shared list data of the cooperative organization, based on the credibility level of the cooperative organization, select the data backtracking amount corresponding to the credibility level from the successfully matched list data for screening.

[0088] The shared list data of the partner institutions involves the relevant subject information provided by other institutions that have business cooperation relationships with financial institutions. The credit rating of the partner institution reflects its comprehensive credit performance, compliance operation level and risk management capabilities in the past cooperation process. The list data provided by a partner institution with good reputation is relatively more reliable and has a relatively low risk probability; while the list data of a partner institution with poor reputation may have more uncertainty and potential risks. For example, a large financial technology company that has a good reputation in the financial industry for many years, strictly abides by regulatory requirements and has a sound internal risk control system as a partner institution has a higher credibility of its shared list data; on the contrary, a newly established small financial services company with an unclear business model and lack of effective regulatory records as a partner institution may need to treat the list data it provides with more caution.

[0089] According to the credit rating of the cooperative institution, the corresponding data backtracking amount is selected from the successfully matched list data for screening. For cooperative institutions with high credit ratings, due to their relatively high data quality and risk controllability, the workload of screening can be appropriately reduced, such as only backtracking recent new business data or a small proportion of historical business data, focusing on the normal development of cooperative business and consistency verification of business data. For example, when cooperating with a well-known international credit rating agency, it may only be necessary to conduct a simple screening of new customer information in its monthly updated shared list data, and conduct regular reviews of some randomly selected historical customer data to ensure the continued compliance of the cooperative business. For cooperative institutions with lower credit ratings, in order to prevent potential risks, it may be necessary to increase the screening efforts, increase the amount of backtracked data and time range, and even conduct full screening of certain key business data. For example, when cooperating with a small financial intermediary with poor credit, it may be necessary to conduct detailed backtracking screening of all customer data in its shared list data, including verification of customer identity information, review of past transaction records, and cross-comparison with other risk data sources, so as to ensure that the financial institution's own business is not affected by the potential risks of the partner and maintain the security and stability of financial business.

[0090] In some other embodiments of the present invention, after step 13, the method may further include:

[0091] Step 14: When the value representing a specific risk in the screening result is greater than a set threshold, a risk warning is triggered and / or the business process is blocked.

[0092] After a series of operations such as the collection of list data, key field matching, and retrospective screening based on lists of different categories, the corresponding screening results will be obtained. This screening result contains a lot of risk-related information, and the "value representing specific risk" is a manifestation of the quantification of the degree of risk. For example, in the anti-money laundering screening scenario, the specific risk value may be a value calculated by a specific algorithm after comprehensively considering multiple factors such as the transaction amount, transaction frequency, counterparty risk level, and abnormal degree of capital flow; in the credit business risk screening, it may be a value generated by combining the applicant's credit score changes, debt situation, recent overdue times, and other information to measure the size of the credit default risk. Different business scenarios will have different risk factors and corresponding calculation methods to determine the value representing the specific risk. The purpose is to use a relatively intuitive value to reflect the risk level of the business involved by the subject.

[0093] In this step, a reasonable threshold is set based on factors such as the nature of the business, risk tolerance, and past experience. This threshold is equivalent to a risk warning line, which is the boundary between acceptable risks and risks that require special attention and measures. For example, in a scenario where a relatively conservative financial institution conducts high-risk investment business, the risk value threshold for investment projects may be set relatively low. Once the risk value reaches this lower limit, the project is considered to be too risky and needs to be intervened; while for some conventional, low-risk financial services, the threshold may be appropriately increased, because the probability of major risks in such businesses is relatively small, and a certain degree of risk fluctuation can be tolerated. The setting of thresholds helps financial institutions quickly screen out those situations that exceed the normal risk range and may pose a threat to the safety and stability of their own businesses from a large number of businesses, avoiding overreaction or missing real high-risk events.

[0094] When the value representing a specific risk in the screening result is greater than the set threshold, it means that the business risk involved in the current screening object has exceeded the acceptable range preset by the institution, and timely response measures are required.

[0095] The purpose of risk warning is to promptly notify relevant departments and personnel (such as risk monitoring teams, business operators, etc.) of high-risk situations so that they can quickly pay attention to and further analyze and assess risks, and initiate corresponding risk response plans. Warnings can be issued in a variety of ways, such as popping up a warning box in the internal system, sending emails or text messages to the terminal devices of relevant persons in charge, prompting specific risk information (such as the subject involved, business type, risk value, and approximate risk points, etc.), so that relevant personnel can know and prepare for response work as soon as possible.

[0096] For some situations where the risk is extremely high and may immediately cause significant losses to financial institutions, a simple warning may not be enough to avoid the occurrence of risks. At this time, it is necessary to block the business process. For example, when it is found that the risk value of a certain ongoing cross-border transfer transaction far exceeds the threshold and is suspected of involving money laundering activities, the system will automatically suspend the transaction, prevent the funds from continuing to flow, avoid the successful transfer of illegal funds, control the risk at the current stage, and prevent further losses. Blocking the business process can be achieved through technical means such as the system's built-in rule engine and permission control to ensure that key business links can be stopped in time under high-risk conditions to ensure the security and compliance of financial services.

[0097] Step 14: By setting up a reasonable risk assessment mechanism and corresponding response strategies, financial institutions can promptly detect and effectively control high-risk situations that exceed acceptable ranges when faced with complex and changing business risks, thereby maintaining the stability and security of financial operations.

[0098] The technical solution provided by the present invention, on the one hand, can ensure the comprehensiveness and diversity of the data by collecting list data from multiple data sources, covering more different categories of information, and by extracting and matching the key fields of each category, it helps to improve the accuracy and pertinence of the data and reduce the possibility of misjudgment and missed judgment. On the second hand, the technical solution adopts automated and intelligent means to process and match data, which can significantly improve screening efficiency and reduce the cost and time of manual intervention. On the third hand, the technical solution also has high flexibility and can adjust the matching rules and backtracking strategies according to different screening needs and scenarios to adapt to different application scenarios.

[0099] Embodiment 2

[0100] See also Figure 4 , Figure 4 : is a schematic diagram of a structure of a device for screening a massive list provided in Embodiment 2 of the present invention, the device comprising:

[0101] A collection module 41 is used to collect list data from multiple data sources, where the list data includes multiple different categories, and extract key fields included in the list data of each category;

[0102] A matching module 42, for matching the key field to be screened with the key field included in the list data according to the matching rule corresponding to the key field to be screened, and determining the category corresponding to the list data to which the key field to be screened belongs;

[0103] The backtracking module 43 is used to determine a backtracking strategy according to the category of the list data to which the key field to be screened belongs, and screen the key field to be screened from the successfully matched list data according to the backtracking strategy to obtain a screening result.

[0104] In some embodiments of the present invention, the collection module 41 may include:

[0105] The collection unit 411 is used to collect list data from multiple data sources, where the list data includes blacklists, warning lists, and shared lists of cooperative organizations.

[0106] The standardization unit 412 is used to standardize the formats of the list data of the multiple data sources to obtain key fields included in the list data of each category.

[0107] In some other embodiments of the present invention, the collection module 41 may further include:

[0108] The deduplication unit 413 is used to remove duplicate data in the list data of the multiple data sources according to the key field.

[0109] The verification unit 414 is used to compare and verify the list data after deduplication with the authoritative data source.

[0110] The correction unit 415 is used to correct the list data with abnormal verification results.

[0111] In some embodiments of the present invention, the key fields include key fields with unique attributes and key fields with diverse attributes, and the matching module 42 may include:

[0112] The first matching unit 421 is used to use an exact matching rule for a key field with a unique attribute to match the key field to be screened with the key field included in the list data. When the key field to be screened is completely consistent with the key field in the list data, it is determined that the match is successful, and the category corresponding to the list data to which the key field to be screened belongs is determined according to the matching result.

[0113] The second matching unit 422 is used to adopt fuzzy matching rules for key fields with diverse attributes to match the key fields to be screened with the key fields included in the list data. When the matching value between the key fields to be screened and the key fields in the list data is greater than a set threshold, the match is determined to be successful, and the category corresponding to the list data to which the key fields to be screened belong is determined based on the matching result.

[0114] In some embodiments of the present invention, the backtracking module 43 may include:

[0115] The first screening unit 431 is used to screen all the list data that have successfully matched when the list data to which the key field to be screened belongs is blacklist data.

[0116] The second screening unit 432 is used for selecting data tracebacks corresponding to the risk level from the successfully matched list data for screening according to the risk level of the warning list data when the list data to which the key field to be screened belongs is the warning list data;

[0117] The third screening unit 433 is used to select the data backtracking amount corresponding to the credibility level from the successfully matched list data for screening according to the credibility level of the cooperative organization when the list data described in the key field to be screened is the shared list data of the cooperative organization.

[0118] In other embodiments of the present invention, the device may further include an alarm module 44 for triggering a risk warning and / or blocking a business process when the value representing a specific risk in the screening result is greater than a set threshold.

[0119] The technical solution provided by the present invention, on the one hand, can ensure the comprehensiveness and diversity of the data by collecting list data from multiple data sources, covering more different categories of information, and by extracting and matching the key fields of each category, it helps to improve the accuracy and pertinence of the data and reduce the possibility of misjudgment and missed judgment. On the second hand, the technical solution adopts automated and intelligent means to process and match data, which can significantly improve screening efficiency and reduce the cost and time of manual intervention. On the third hand, the technical solution also has high flexibility and can adjust the matching rules and backtracking strategies according to different screening needs and scenarios to adapt to different application scenarios.

[0120] It should be noted that the massive list screening device in the embodiment of the present invention and the massive list screening method in the above embodiment belong to the same inventive concept. The technical details not described in detail in this device can be found in the previous description of the method and will not be repeated here.

[0121] In addition, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the aforementioned method when running.

[0122] Figure 5 A schematic diagram of an electronic device 50 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0123] like Figure 5 As shown, the electronic device 50 includes at least one processor 51, and a memory connected to the at least one processor 51 in communication, such as a read-only memory (ROM) 52, a random access memory (RAM) 53, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 51 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 52 or the computer program loaded from the storage unit 58 to the random access memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the electronic device 50 can also be stored. The processor 51, the ROM 52, and the RAM 53 are connected to each other through a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0124] A number of components in the electronic device 50 are connected to the I / O interface 55, including: an input unit 56, such as a keyboard, a mouse, etc.; an output unit 57, such as various types of displays, speakers, etc.; a storage unit 58, such as a disk, an optical disk, etc.; and a communication unit 59, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0125] The processor 51 may be a variety of general and / or dedicated processing components with processing and computing capabilities. Some examples of the processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 51 executes the various methods and processes described above, such as an idle detection method.

[0126] In some embodiments, the idle detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 50 via the ROM 52 and / or the communication unit 59. When the computer program is loaded into the RAM 53 and executed by the processor 51, one or more steps of the idle detection method described above may be performed. Alternatively, in other embodiments, the processor 51 may be configured to perform the idle detection method in any other appropriate manner (e.g., by means of firmware).

[0127] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0128] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0129] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0130] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0131] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0132] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0133] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0134] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for screening a massive list, characterized in that: The method comprises: Collecting list data from multiple data sources, wherein the list data includes multiple different categories, and extracting key fields included in the list data of each category; According to the matching rules corresponding to the key fields to be screened, the key fields to be screened are matched with the key fields included in the list data to determine the category corresponding to the list data to which the key fields to be screened belong; A backtracking strategy is determined according to the category of the list data to which the key field to be screened belongs, and the key field to be screened is screened from the successfully matched list data according to the backtracking strategy to obtain a screening result.

2. The method according to claim 1, characterized in that Collect list data from multiple data sources, the list data contains multiple different categories, and extract key fields contained in the list data of each category including: Collecting list data from multiple data sources, the list data categories include blacklists, warning lists and shared lists of cooperative institutions; The formats of the list data of the multiple data sources are standardized to obtain key fields contained in the list data of each category.

3. The method according to claim 2, characterized in that The method further comprises: Remove duplicate data from the list data of the multiple data sources according to the key fields; Compare and verify the list data after deduplication with the authoritative data source; Correct the list data with abnormal verification results.

4. The method according to claim 1, characterized in that: The key fields include key fields with unique attributes and key fields with diverse attributes; According to the matching rules corresponding to the key fields to be screened, the key fields to be screened are matched with the key fields included in the list data, and the categories corresponding to the list data to which the key fields to be screened belong are determined to include: An exact matching rule is adopted for key fields with unique attributes to match the key fields to be screened with the key fields contained in the list data. When the key fields to be screened are completely consistent with the key fields in the list data, the match is determined to be successful, and the category corresponding to the list data to which the key fields to be screened belong is determined based on the matching result; Fuzzy matching rules are used for key fields with diverse attributes to match the key fields to be screened with the key fields contained in the list data. When the matching value between the key fields to be screened and the key fields in the list data is greater than the set threshold, the match is determined to be successful, and the category corresponding to the list data to which the key fields to be screened belong is determined based on the matching results.

5. The method according to claim 2, characterized in that: Determining a backtracking strategy according to the category of the list data to which the key field to be screened belongs, and screening the key field to be screened from the successfully matched list data according to the backtracking strategy includes: When the list data to which the key field to be screened belongs is blacklist data, screening is performed from all the list data that successfully matches; When the list data to which the key field to be screened belongs is the warning list data, according to the risk level of the warning list data, the data backtracking amount corresponding to the risk level is selected from the successfully matched list data for screening; When the list data described in the key field to be screened is the shared list data of a cooperative organization, based on the credibility level of the cooperative organization, the data backtracking amount corresponding to the credibility level is selected from the successfully matched list data for screening.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: When the value representing a specific risk in the screening result is greater than a set threshold, a risk warning is triggered and / or the business process is blocked.

7. A device for screening a large number of lists, characterized in that: The device comprises: A collection module, used to collect list data from multiple data sources, wherein the list data includes multiple different categories, and extract key fields included in the list data of each category; A matching module, used for matching the key field to be screened with the key field included in the list data according to the matching rules corresponding to the key field to be screened, and determining the category corresponding to the list data to which the key field to be screened belongs; The backtracking module is used to determine a backtracking strategy according to the category of the list data to which the key field to be screened belongs, and screen the key field to be screened from the successfully matched list data according to the backtracking strategy to obtain a screening result.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method for screening a massive list according to any one of claims 1 to 6 when running.

9. An electronic device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method for screening a massive list according to any one of claims 1 to 6.

Images