List data filtering method and device, electronic equipment and readable storage medium
By identifying and reassigning the weights of invalid attributes during list screening, and combining the matching rate to calculate the list data weights, the problem of low screening accuracy in existing technologies is solved, and more efficient list data screening is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING PACTERA JINXIN TECH LTD
- Filing Date
- 2022-08-25
- Publication Date
- 2026-05-29
AI Technical Summary
Existing list screening methods rely solely on matching degree, resulting in low screening accuracy and an inability to effectively distinguish invalid attributes, thus affecting the screening effect of list data.
By determining the target attributes and invalid attributes under the target screening conditions, the initial weights of invalid attributes are reassigned, the list data weights of the candidate list data are calculated, and the final list data is calculated using the matching rate and weights of the valid attributes.
It improves the accuracy of list data screening, ensures that the screened list data is more valuable for application, and reduces the impact of invalid attributes.
Smart Images
Figure CN115374322B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and more specifically, to a list data filtering method, apparatus, electronic device, and readable storage medium. Background Technology
[0002] With the development of computer technology, more and more business operations are being moved online. Many of these operations require list screening. For example, when performing a certain task, it's necessary to first determine whether the data to be processed is on a blacklist or whitelist, and then decide whether to proceed with the next step based on the screening results.
[0003] Current list screening methods are relatively simple, generally involving a simple match between screening criteria and the attributes of the list data, determining whether the list data matches based on the degree of match. However, the attributes corresponding to the matched list data determined solely by the degree of match may contain invalid attributes that do not meet the valid attribute conditions. This makes it difficult to accurately determine valid matched list data by directly using the degree of match, thus affecting the accuracy of list data screening. Summary of the Invention
[0004] The purpose of this application is to solve the problem of low accuracy in screening list data.
[0005] This application provides a method for filtering list data, the method comprising:
[0006] At least one candidate list data is determined from the list to be filtered based on the target screening conditions;
[0007] Determine at least one target attribute corresponding to the target screening condition;
[0008] Based on at least one target attribute, invalid attributes that do not match at least one target attribute are determined from multiple preset attributes; wherein each preset attribute is set with a corresponding initial weight;
[0009] Based on the initial weights of invalid attributes, the attribute weights of at least one target attribute are reassigned to obtain the first attribute weights corresponding to each target attribute.
[0010] Based on the weight of the first attribute corresponding to each target attribute, the list data weight of each candidate list data is obtained, and the valid list data is determined from at least one candidate list data according to the list data weight.
[0011] In an alternative embodiment of the first aspect, the method further includes:
[0012] Set the initial weight for each preset attribute, and the sum of the initial weights of the preset attributes is 1.
[0013] In an alternative embodiment of the first aspect, reallocating the attribute weights of at least one target attribute based on the initial weights of invalid attributes includes:
[0014] The initial weights of the invalid attributes are summed to obtain the weight values to be assigned.
[0015] The attribute weights of at least one target attribute are reassigned based on the weight values to be assigned.
[0016] In an optional embodiment of the first aspect, reallocating the attribute weights of at least one target attribute according to the weight values to be assigned includes:
[0017] Determine the proportional relationship between the attribute weights of at least one target attribute, and reallocate the weights to be assigned to the attribute weights of at least one target attribute according to the proportional relationship; the sum of the first attribute weights of at least one target attribute after allocation is 1.
[0018] In an optional embodiment of the first aspect, each target attribute is set with corresponding valid attribute conditions, and the list data weights of each candidate list data are obtained based on the first attribute weights corresponding to each target attribute, including:
[0019] Based on the valid attribute condition, a valid target attribute is determined from at least one target attribute, and attributes other than the valid target attribute among the at least one target attribute are regarded as invalid target attributes;
[0020] Based on the invalid target attributes, the weights of the first attributes corresponding to each valid target attribute are redistributed to obtain the weights of the second attributes of each valid target attribute.
[0021] For each candidate list data, at least one valid target attribute corresponding to the candidate list data is obtained, the matching rate of each valid target attribute is determined, and the list data weight of the candidate list data is calculated based on the matching rate of each valid target attribute and the weight of the second attribute. The matching rate is used to characterize the degree of matching between the valid target attribute and the target screening conditions.
[0022] In an optional embodiment of the first aspect, the list data weight of the candidate list data is calculated based on the matching rate of each corresponding valid target attribute and the weight of the second attribute, including:
[0023] Multiply the matching rate of each valid target attribute by the weight of the second attribute to obtain the target matching rate of each valid target attribute; sum the target matching rates of each valid target attribute to obtain the list data weight of the candidate list data.
[0024] In an optional embodiment of the first aspect, determining valid list data from at least one candidate list data based on list data weights includes:
[0025] Obtain the preset weight threshold for the list data;
[0026] Candidate list data whose list data weight is greater than the list data weight threshold are considered valid list data.
[0027] Secondly, a list data filtering device is provided, the device comprising:
[0028] The list data screening module is used to determine at least one candidate list data from the list to be filtered based on the target screening conditions.
[0029] The invalid attribute determination module is used to determine at least one target attribute corresponding to the target screening condition; based on the at least one target attribute, it determines invalid attributes that do not match the at least one target attribute from multiple preset attributes; wherein, each preset attribute is set with a corresponding initial weight;
[0030] The weight reallocation module is used to reallocate the attribute weights of at least one target attribute based on the initial weights of invalid attributes, so as to obtain the first attribute weights corresponding to each target attribute.
[0031] The list data determination module is used to obtain the list data weight of each candidate list data based on the first attribute weight corresponding to each target attribute, and determine the valid list data from at least one candidate list data according to the list data weight.
[0032] Thirdly, an electronic device is provided, the electronic device comprising:
[0033] A memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the list data filtering method of any of the above embodiments.
[0034] Fourthly, a readable storage medium is provided, on which a program is stored, which, when executed by a processor, implements the list data filtering method of any of the above embodiments.
[0035] The aforementioned list data filtering method includes: determining at least one candidate list data from the list to be filtered based on target screening conditions; then, based on at least one target attribute corresponding to the target screening conditions, determining invalid attributes from multiple preset attributes that do not match at least one target attribute. The attribute weights of each target attribute can be increased based on the initial weights of the invalid attributes to obtain the first attribute weights corresponding to each target attribute. Then, the list data weights of each candidate list data are obtained based on the first attribute weights corresponding to each target attribute. Finally, the valid list data is determined by judging whether the list data weights are greater than a preset list data weight threshold, thus improving the accuracy of list data screening and making the screened list data more valuable for application. Attached Figure Description
[0036] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.
[0037] Figure 1 A flowchart illustrating a list data filtering method provided in an embodiment of this application;
[0038] Figure 2 A flowchart illustrating a list data filtering method provided in an embodiment of this application;
[0039] Figure 3 A flowchart illustrating a list data filtering method provided in an embodiment of this application;
[0040] Figure 4 This is a schematic diagram of the structure of a list data filtering device provided in an embodiment of this application;
[0041] Figure 5 This is a schematic diagram of the structure of an electronic device for filtering list data, provided as an embodiment of this application. Detailed Implementation
[0042] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.
[0043] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”
[0044] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0045] With the development of computer technology, more and more business operations are being moved online. Many of these operations require list screening. For example, when performing a certain task, it's necessary to first determine whether the data to be processed is on a blacklist or whitelist, and then decide whether to proceed with the next step based on the screening results.
[0046] Current list screening methods are relatively simple. Generally, they involve simply matching the screening criteria with the attributes of the list data, and determining whether the list data matches based on the degree of matching.
[0047] However, in the business application of list screening, different business channels may have different screening conditions for different types of lists. Moreover, if an attribute corresponding to a certain screening condition is not present in the list that has been hit, the processing requirements of different channels may also be different. The attributes corresponding to the hit list data determined solely by the degree of matching may contain invalid attributes that do not meet the valid attribute conditions, making it difficult to accurately determine the hit list data by directly using the degree of matching, thus affecting the accuracy of list data screening.
[0048] The list data filtering method, apparatus, electronic device, and readable storage medium provided in this application are intended to solve the above-mentioned technical problems of the prior art.
[0049] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.
[0050] This application provides a method for filtering list data, such as... Figure 1 As shown, the method includes:
[0051] Step S101: Determine at least one candidate list data from the list to be filtered based on the target screening conditions.
[0052] The list data filtering method provided in this application can be applied to any business system that has list data filtering requirements.
[0053] Different business systems for different business scenarios can correspond to different lists to be filtered and different screening conditions. Within the same business system, multiple screening conditions can also be available for different business needs; this application does not impose any restrictions on this.
[0054] The target screening criteria can be determined from a variety of screening criteria, and at least one candidate list data can be determined from the filter list based on the target screening criteria.
[0055] This application does not limit the specific screening method based on the target screening conditions. As an example, the following scheme is provided: the target screening conditions can correspond to at least one target attribute and the target attribute value of each target attribute. At least one candidate list data can be determined from the list to be filtered based on the target attribute value of each target attribute.
[0056] Each candidate list data item contains all target attributes, and the target attribute values in the candidate list data match the target attribute values in the target screening conditions. The matching determination method can be based on a preset matching rate threshold; when the matching rate between the target attribute values in the candidate list data and the target attribute values in the screening conditions is greater than the matching rate threshold, the two are considered to be matched.
[0057] For example, the target attributes in the target screening criteria can be "name," "address," and "account." The attributes of the candidate list data determined based on the target screening criteria can also include these target attributes. Suppose the attribute values for "name," "address," and "account" in the target screening criteria are "Beijing Co., Ltd.", "Beijing," and "beijing," respectively, and the attribute values for "name," "address," and "account" in the candidate list data are "Beijing Happy Co., Ltd.", "Beijing Sanlitun," and "beijingkuaile," respectively. For any target attribute, such as "name," the matching rate between "Beijing Co., Ltd." in the target screening criteria and "Beijing Happy Co., Ltd." in the candidate list data meets the preset matching rate threshold; therefore, they can be considered a match. Similarly, if the attribute values of the other two target attributes also match, the candidate list data is determined.
[0058] The matching rate can be calculated by measuring the similarity between two strings, and this application does not impose any restrictions.
[0059] Step S102: Determine at least one target attribute corresponding to the target screening condition; based on the at least one target attribute, determine invalid attributes from multiple preset attributes that do not match the at least one target attribute; wherein, each preset attribute is set with a corresponding initial weight.
[0060] In the embodiments of this application, the target screening condition may correspond to at least one target attribute. Based on at least one target attribute, invalid attributes that do not match at least one target attribute can be determined from multiple preset attributes. Here, mismatch may refer to different attribute names, and invalid attributes may refer to attributes among the preset attributes whose names are different from any of the target attributes.
[0061] This means that any other attribute in the preset properties that has the same name as at least one target attribute can be considered an invalid attribute. Alternatively, it can be understood as any other attribute in the preset properties that is not the target attribute.
[0062] Each preset attribute has a corresponding initial weight.
[0063] In one example, the preset attributes may include "attribute 1", "attribute 2", "attribute 3", "attribute 4", "attribute 5", and "attribute 6", and the target attributes may include "attribute 1", "attribute 2", and "attribute 6". In this case, "attribute 3", "attribute 4", and "attribute 5" can be considered invalid attributes.
[0064] In this embodiment of the application, the initial weight corresponding to each preset attribute can be preset, and the sum of the initial weights of the preset attributes is 1.
[0065] For example, if there are preset attributes "Attribute 1", "Attribute 2", "Attribute 3", "Attribute 4", "Attribute 5", and "Attribute 6", the initial weights of each preset attribute can be set to "0.1", "0.2", "0.05", "0.25", "0.1", and "0.4" respectively. The sum of the initial weights of all preset attributes is 1.
[0066] In some embodiments, the initial weights of each preset attribute of different business systems in different business scenarios may be different, and the types of preset attributes may also be different. This application does not impose any restrictions on this.
[0067] In some embodiments, different lists to be filtered can be different categories, and preset attributes applicable to filtering the lists to be filtered and the initial weights of each preset attribute can be determined based on the category of the lists to be filtered.
[0068] Step S103: Based on the initial weights of invalid attributes, the attribute weights of at least one target attribute are reassigned to obtain the first attribute weights corresponding to each target attribute.
[0069] In this embodiment of the application, each preset attribute is set with a corresponding initial weight. After determining the invalid attribute in the preset attributes, the attribute weight of each target attribute can be reallocated according to the initial weight of the invalid attribute in the preset attributes to obtain the first attribute weight corresponding to each target attribute.
[0070] In this embodiment of the application, reallocating the attribute weight of at least one target attribute based on the initial weight of an invalid attribute may include the following steps:
[0071] (1) Sum the initial weights of the invalid attributes to obtain the weight values to be assigned. For example, the preset attributes include "Attribute 1", "Attribute 2" and "Attribute 3", and the initial weights of the above attributes can be "0.5", "0.3" and "0.2" respectively. Assuming that "Attribute 1" does not belong to the target attribute, "Attribute 1" can be regarded as an invalid attribute, and the weight value to be assigned is "0.5".
[0072] (2) Reassign the attribute weight of at least one target attribute according to the weight value to be assigned.
[0073] Specifically, the weight value to be assigned can be split into multiple parts and assigned to each target attribute. For example, when "Attribute 1" is an invalid attribute and the weight value to be assigned is "0.5", "0.5" can be reassigned to the two target attributes "Attribute 2" and "Attribute 3".
[0074] In this embodiment of the application, reallocating the attribute weights of at least one target attribute according to the weight values to be assigned may include the following steps:
[0075] Determine the proportional relationship between the attribute weights of at least one target attribute, and reallocate the weights to be assigned to the attribute weights of at least one target attribute according to the proportional relationship; the sum of the attribute weights of at least one target attribute after allocation is 1.
[0076] For example, the preset attributes include "Attribute 1", "Attribute 2", and "Attribute 3", and the initial weights of these attributes can be "0.5", "0.3", and "0.2", respectively. Assuming that "Attribute 1" does not belong to the target attribute, "Attribute 1" can be regarded as an invalid attribute, and the weight value to be assigned is "0.5".
[0077] The ratio between the attribute weights of "Attribute 2" and "Attribute 3" is 3:2. Therefore, the weight value "0.5" to be assigned can be allocated to "Attribute 2" and "Attribute 3" according to the ratio. That is, "0.3" of "0.5" is allocated to "Attribute 2" and "0.2" is allocated to "Attribute 3". After the reallocation, the first attribute weights of "Attribute 2" and "Attribute 3" are "0.6" and "0.4" respectively, and the sum is still 1.
[0078] Step S104: Obtain the list data weight of each candidate list data based on the first attribute weight corresponding to each target attribute, and determine the valid list data from at least one candidate list data according to the list data weight.
[0079] In this embodiment, the list data weight is used to characterize the validity of the candidate list data. The list data weight of each candidate list data can be obtained according to the first attribute weight corresponding to each target attribute, and the validity of each candidate list data can be determined according to the list data weight, thereby determining the valid list data and improving the accuracy of list data filtering.
[0080] In this embodiment of the application, determining valid list data from at least one candidate list data based on list data weight may include the following steps: obtaining a preset list data weight threshold; and selecting candidate list data whose list data weight is greater than the list data weight threshold as valid list data.
[0081] The aforementioned list data filtering method includes: determining at least one candidate list data from the list to be filtered based on target screening conditions; then, based on at least one target attribute corresponding to the target screening conditions, determining invalid attributes from multiple preset attributes that do not match at least one target attribute. The attribute weights of each target attribute can be increased based on the initial weights of the invalid attributes to obtain the first attribute weights corresponding to each target attribute. Then, the list data weights of each candidate list data are obtained based on the first attribute weights corresponding to each target attribute. Finally, the valid list data is determined by judging whether the list data weights are greater than a preset list data weight threshold, thus improving the accuracy of list data screening and making the screened list data more valuable for application.
[0082] This application provides a possible implementation method where each target attribute can be set with corresponding valid attribute conditions. This application does not limit the specific form and content of the valid attribute conditions. For example, for attributes with numeric values, the valid attribute condition can be a range of attribute values; for attributes with field values, the valid attribute condition can be a preset keyword or require the field of the attribute value to contain preset content. In one example, there is a target attribute "address," and its corresponding valid attribute condition could be requiring the attribute value of "address" to contain a location name below the country level (e.g., province or state). Suppose that the attribute value of the "address" in a list of data only contains "country A" and does not contain a location name below the country level; then the target attribute "address" is an invalid target attribute for that list of data.
[0083] Specifically, obtaining the list data weights of each candidate list data based on the weights of the first attribute corresponding to each target attribute can include the following steps:
[0084] (1) For each candidate list data, determine the valid target attribute corresponding to the candidate list data from at least one target attribute based on the valid attribute condition, and take the attribute other than the valid target attribute in the at least one target attribute as the invalid target attribute corresponding to the candidate list data.
[0085] Specifically, since the attribute values of each candidate list data for the same target attribute may be different, some candidate list data may meet the valid attribute conditions for the target attribute, while other candidate list data may not meet the valid attribute conditions for the target attribute. Therefore, the valid target attributes and invalid target attributes corresponding to each candidate list data may be different.
[0086] (2) Based on the invalid target attributes, the first attribute weights corresponding to each valid target attribute are redistributed to obtain the second attribute weights of each valid target attribute corresponding to the candidate list data.
[0087] The method for reallocating the first attribute weights corresponding to each valid target attribute based on invalid target attributes can be the same as the method for obtaining the first attribute weights through the first reallocation, specifically including the following steps:
[0088] The first attribute weights of invalid target attributes are summed to obtain the first weight value. The proportional relationship between the first attribute weights of each valid target attribute is determined, and the first weight values are redistributed to each valid target attribute according to the proportional relationship to obtain the second attribute weights of each valid target attribute corresponding to the candidate list data.
[0089] (3) Obtain the matching rate of each effective target attribute corresponding to the candidate list data, and calculate the list data weight of the candidate list data based on the matching rate of each effective target attribute and the weight of the second attribute; the matching rate is used to characterize the degree of matching between the effective target attribute and the target screening conditions.
[0090] Specifically, the matching rate of any valid target attribute in the candidate list data can be defined as the matching rate between the attribute value of a valid target attribute in the candidate list data and the attribute value of a valid target attribute in the target screening conditions. The target screening conditions include at least one reference attribute value for a target attribute (i.e., a reference value used for screening), and the valid target attribute is determined from at least one target attribute. Therefore, the target screening conditions naturally include the reference attribute value of the valid target attribute.
[0091] In the embodiments of this application, the similarity between the reference attribute value of the effective target attribute in the target screening conditions and the attribute value of the effective target attribute in the candidate list data can be calculated, and the similarity can be used as the matching rate.
[0092] In this embodiment of the application, the list data weight of the candidate list data is calculated based on the matching rate of each corresponding valid target attribute and the weight of the second attribute, including:
[0093] Multiply the matching rate of each valid target attribute by the weight of the second attribute to obtain the target matching rate of each valid target attribute; sum the target matching rates of each valid target attribute to obtain the list data weight of the candidate list data.
[0094] Specifically, after obtaining the matching rate of each valid target attribute corresponding to the candidate list data, the list data weight can be calculated based on the matching rate of each valid target attribute and the weight of the second attribute. Specifically, for each valid target attribute, the matching rate of that valid target attribute is multiplied by the weight of the second attribute to obtain the target matching rate of that target attribute. Then, the target matching rates of all valid target attributes corresponding to that candidate list data are summed to obtain the list data weight. After redistributing the attribute weights twice using the above method, the final calculated list data weight can more accurately assess the effectiveness of the list data.
[0095] In one example, assuming the effective target attributes corresponding to the candidate list data are “attribute 1”, “attribute 2” and “attribute 3”, the matching rates of each effective target attribute are “100%”, “50%” and “100%” respectively, and the weights of the second attribute are “0.375”, “0.5” and “0.125” respectively, the list data weight of the candidate list data can be calculated as 0.75 by formula (1).
[0096] 0.375*1.0+0.5*0.5+0.125*1.0=0.375+0.25+0.125=0.75 formula (1)
[0097] In one example, the list data filtering method provided in this application is as follows: Figure 2 As shown, it may include the following steps:
[0098] Step S201: Determine at least one candidate list data from the list to be filtered based on the target screening conditions;
[0099] Step S202: Determine at least one target attribute corresponding to the target screening condition; based on the at least one target attribute, determine invalid attributes that do not match the at least one target attribute from multiple preset attributes; wherein, each preset attribute is set with a corresponding initial weight, and the sum of the initial weights of the preset attributes is 1.
[0100] Step S203: Sum the initial weights of the invalid attributes to obtain the weight values to be assigned;
[0101] Step S204: Determine the proportional relationship between the attribute weights of at least one target attribute, and reallocate the weights to be assigned to the attribute weights of at least one target attribute according to the proportional relationship to obtain the first attribute weights corresponding to each target attribute; the sum of the first attribute weights of the at least one target attribute after allocation is 1.
[0102] Step S205: Based on the valid attribute condition, determine the valid target attribute from at least one target attribute, and treat the attributes other than the valid target attribute among the at least one target attribute as invalid target attributes;
[0103] Step S206: Based on the invalid target attributes, the first attribute weights corresponding to each valid target attribute are redistributed to obtain the second attribute weights of each valid target attribute;
[0104] Step S207: For each candidate list data, obtain at least one valid target attribute corresponding to the candidate list data, determine the matching rate of each valid target attribute, and calculate the list data weight of the candidate list data based on the matching rate of each valid target attribute and the weight of the second attribute.
[0105] Step S208: Obtain a preset list data weight threshold, and take candidate list data whose list data weight is greater than the list data weight threshold as valid list data.
[0106] In practical applications, the list data filtering method provided in this application can be applied to list management and monitoring platforms to accurately screen lists.
[0107] A list can be understood as a list of sanctioned entities or individuals. These entities or individuals are typically identified using attributes such as name, address, identification document, country, and account number for screening and matching. The list and the attribute values corresponding to each attribute can be used as list data. Specifically, list data filtering methods may include the following steps:
[0108] The first step is to configure the weighting rules. The weighting rules can contain the initial weight values for all attributes within the system, and the sum of the initial weight values for all attributes is 1.
[0109] The second step involves determining the target screening criteria based on the requirements, screening the list to be filtered, and identifying at least one candidate list from it. Simultaneously, the attributes corresponding to the target screening criteria are determined, and then it is judged which attributes in the weighting rules correspond to the target screening criteria and which attributes are irrelevant. The target screening criteria may include reference attribute values for multiple attributes.
[0110] Attributes irrelevant to the screening criteria can be removed as invalid attributes. Then, the initial weights of these invalid attributes are summed and redistributed proportionally to the target attributes corresponding to the target screening criteria, ensuring that the sum of the first attribute weights of the redistributed target attributes remains 1. Here, the proportional relationship refers to the ratio of the initial weights of each target attribute.
[0111] In one example, the preset attributes include "Attribute 1", "Attribute 2", and "Attribute 3", with initial weights of "0.5", "0.3", and "0.2" respectively. Assuming "Attribute 1" is not a target attribute, it can be considered an invalid attribute with a weight of "0.5".
[0112] The ratio between the attribute weights of "Attribute 2" and "Attribute 3" is 3:2. Therefore, the weight value "0.5" to be assigned can be allocated to "Attribute 2" and "Attribute 3" according to the ratio. That is, "0.3" of "0.5" is allocated to "Attribute 2" and "0.2" is allocated to "Attribute 3". After the reallocation, the first attribute weights of "Attribute 2" and "Attribute 3" are "0.6" and "0.4" respectively, and the sum is still 1.
[0113] The third step, in addition to determining which attributes are valid and which are invalid according to the weighting rules, can also determine the validity of each attribute value in each candidate list data.
[0114] Specifically, each target attribute is set with corresponding valid attribute conditions. For each candidate list data, the validity of each attribute value of the candidate list data can be determined based on the valid attribute conditions. The target attribute corresponding to the attribute value that meets the valid attribute conditions is taken as the valid target attribute of the candidate list data, and the target attribute corresponding to the attribute value that does not meet the valid attribute conditions is taken as the invalid target attribute of the candidate list data.
[0115] The fourth step is to redistribute the first attribute weights corresponding to each valid target attribute based on the invalid target attributes, thereby obtaining the second attribute weights of each valid target attribute corresponding to the candidate list data.
[0116] The fifth step is to obtain the matching rate of each valid target attribute corresponding to the candidate list data, and calculate the list data weight of the candidate list data based on the matching rate of each valid target attribute and the weight of the second attribute.
[0117] Specifically, the matching rate of each valid target attribute can be multiplied by the weight of the second attribute to obtain the target matching rate of each valid target attribute; the target matching rates of each valid target attribute can be summed to obtain the list data weight of the candidate list data.
[0118] The sixth step is to obtain the preset list data weight threshold and select candidate list data whose list data weight is greater than the list data weight threshold as valid list data.
[0119] In one example, the list data filtering method provided in this application may include, for example: Figure 3 The steps shown are as follows:
[0120] The first step is to parse the weighting rules to obtain the attribute weight of each weighted attribute;
[0121] The second step is to determine whether all weighted attributes have corresponding screening conditions. If so, the hit list information is obtained; otherwise, weighted attributes that do not have corresponding screening conditions are ignored, the attribute weights of other weighted attributes are increased, and then the hit list information is obtained.
[0122] The third step is to determine whether all attribute values in the hit list are valid. If so, multiply the attribute weight and the attribute hit rate, and sum them up. Otherwise, increase the weight of the invalid attributes, increase the weight of the other attributes, multiply the attribute weight and the attribute hit rate, and sum them up.
[0123] The fourth step is to determine whether the summed value reaches the total weight of the preset weighting rule. If so, the hit list information is retained; otherwise, the hit list information is discarded.
[0124] In one example, the preset weighting rule can be expressed as the following expression (1).
[0125] [NAME:0.3][CREDENTIAL:0.4][ADDRESS:0.2][COUNTRY:0.1]>=0.9
[0126] Expression (1)
[0127] The weighting rule includes four attributes: name, identification document, address, and country, with initial weights of "0.3", "0.4", "0.2", and "0.1" for each attribute, respectively. The 0.9 on the right indicates that the weight threshold for the list data is 0.9.
[0128] In actual business operations, the screening conditions of a certain interface may not include the address attribute. In this case, the address is an invalid attribute. The attribute weights of the other three items (name, document and country) can be redistributed based on the initial weight and proportional relationship of the address. The redistribution method can be expressed as expressions (2) to (4).
[0129] [NAME:(0.2*0.3 / (0.3+0.4+0.1)+0.3)] Expression (2)
[0130] [CREDENTIAL:(0.2*0.4 / (0.3+0.4+0.1)+0.4)] Expression (3)
[0131] [COUNTRY:(0.2*0.1 / (0.3+0.4+0.1)+0.1)] Expression (4)
[0132] The calculation results are as follows: the weight of the first attribute of the name is 0.375, the weight of the first attribute of the document is 0.5, and the weight of the first attribute of the country is 0.125.
[0133] The data in the list to be filtered can be matched based on the name, identification document, and country in the screening criteria. Assuming the final count yields three matching lists with numbers 001, 002, and 003, list 001 has a 90% name match rate (but lacks valid identification information) and a 100% country match rate. Because list 001 lacks valid identification information, the weights need further adjustment. The adjusted weights are: 0.8 for the second attribute of the name and 0.2 for the second attribute of the country. The method for the second weight allocation is similar to the first.
[0134] After obtaining the weight of the second attribute and the matching rate, the weight of the list data for the 001 list can be calculated. The calculation formula and the calculation result are shown in formula (2).
[0135] 0.8*0.9+0.2*1.0=0.72+0.2=0.92 Formula (2)
[0136] Since the weight of the 001 list data is 0.92, which is greater than the list weight threshold of 0.9, the hit result of the 001 list is valid.
[0137] Assuming the name matching rate of the 002 list is 100%, the document matching rate is 50%, and the country matching rate is 100%, since the name, document, and country attributes of the 002 list are all valid, the list data weight of the 002 list can be directly calculated. The calculation formula and calculation result are shown in formula (3).
[0138] 0.375*1.0+0.5*0.5+0.125*1.0=0.375+0.25+0.125=0.75 Formula (3)
[0139] Since the weight of list 002 is 0.75, which is less than the list weight threshold of 0.9, the hit result of list 001 is invalid and is ultimately discarded.
[0140] This application provides a list data filtering device, such as... Figure 4 As shown, the list data filtering device 40 may include: a list data screening module 401, an invalid attribute determination module 402, a weight reallocation module 403, and a list data determination module 404, wherein,
[0141] The list data screening module 401 is used to determine at least one candidate list data from the list to be filtered based on the target screening conditions.
[0142] The invalid attribute determination module 402 is used to determine at least one target attribute corresponding to the target screening condition; based on the at least one target attribute, it determines invalid attributes that do not match the at least one target attribute from multiple preset attributes; wherein, each preset attribute is set with a corresponding initial weight;
[0143] The weight reallocation module 403 is used to reallocate the attribute weight of at least one target attribute based on the initial weight of the invalid attribute, so as to obtain the first attribute weight corresponding to each target attribute.
[0144] The list data determination module 404 is used to obtain the list data weight of each candidate list data based on the first attribute weight corresponding to each target attribute, and determine the valid list data from at least one candidate list data according to the list data weight.
[0145] The aforementioned list data filtering device includes: determining at least one candidate list data from the list to be filtered based on target screening conditions; and then, based on at least one target attribute corresponding to the target screening conditions, determining invalid attributes from multiple preset attributes that do not match at least one target attribute. The device can increase the attribute weight of each target attribute based on the initial weight of the invalid attributes to obtain the first attribute weight corresponding to each target attribute. Then, it obtains the list data weight of each candidate list data based on the first attribute weight corresponding to each target attribute. Finally, it determines the valid list data by judging whether the list data weight is greater than a preset list data weight threshold, thereby improving the accuracy of list data screening and making the screened list data more valuable for application.
[0146] In this embodiment of the application, an initial weight setting module is also included, specifically used for:
[0147] Set the initial weight for each preset attribute, and the sum of the initial weights of the preset attributes is 1.
[0148] In this embodiment of the application, when the weight reallocation module 403 reallocates the attribute weight of at least one target attribute based on the initial weight of an invalid attribute, it is specifically used for:
[0149] The initial weights of the invalid attributes are summed to obtain the weight values to be assigned.
[0150] The attribute weights of at least one target attribute are reassigned based on the weight values to be assigned.
[0151] In this embodiment of the application, when the weight reallocation module 403 reallocates the attribute weights of at least one target attribute according to the weight values to be allocated, it is specifically used for:
[0152] Determine the proportional relationship between the attribute weights of at least one target attribute, and reallocate the weights to be assigned to the attribute weights of at least one target attribute according to the proportional relationship; the sum of the first attribute weights of at least one target attribute after allocation is 1.
[0153] In this embodiment of the application, when the list data determination module 404 sets corresponding valid attribute conditions for each target attribute and obtains the list data weights of each candidate list data based on the first attribute weights corresponding to each target attribute, it is specifically used for:
[0154] Based on the valid attribute condition, a valid target attribute is determined from at least one target attribute, and attributes other than the valid target attribute among the at least one target attribute are regarded as invalid target attributes;
[0155] Based on the invalid target attributes, the weights of the first attributes corresponding to each valid target attribute are redistributed to obtain the weights of the second attributes of each valid target attribute.
[0156] For each candidate list data, at least one valid target attribute corresponding to the candidate list data is obtained, the matching rate of each valid target attribute is determined, and the list data weight of the candidate list data is calculated based on the matching rate of each valid target attribute and the weight of the second attribute. The matching rate is used to characterize the degree of matching between the valid target attribute and the target screening conditions.
[0157] In this embodiment of the application, when the list data determination module 404 calculates the list data weight of the candidate list data based on the matching rate of each corresponding valid target attribute and the weight of the second attribute, it is specifically used for:
[0158] Multiply the matching rate of each valid target attribute by the weight of the second attribute to obtain the target matching rate of each valid target attribute; sum the target matching rates of each valid target attribute to obtain the list data weight of the candidate list data.
[0159] In this embodiment of the application, when the list data determination module 404 determines valid list data from at least one candidate list data according to the list data weight, it is specifically used for:
[0160] Obtain the preset weight threshold for the list data;
[0161] Candidate list data whose list data weight is greater than the list data weight threshold are considered valid list data.
[0162] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.
[0163] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the list data filtering method. Compared with related technologies, this method can improve the accuracy of list data screening and make the screened list data more valuable for application.
[0164] In one alternative embodiment, an electronic device is provided, such as Figure 5 As shown, Figure 5 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.
[0165] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.
[0166] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0167] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.
[0168] The memory 4003 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.
[0169] Electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, and tablets, as well as fixed terminals such as digital TVs and desktop computers.
[0170] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.
[0171] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.
[0172] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.
[0173] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.
[0174] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.
Claims
1. A method for filtering list data, characterized in that, include: At least one candidate list data is determined from the list to be filtered based on the target screening conditions; Determine at least one target attribute corresponding to the target screening condition; Based on the at least one target attribute, invalid attributes that do not match the at least one target attribute are determined from a plurality of preset attributes; wherein each preset attribute is set with a corresponding initial weight; Based on the initial weight of the invalid attribute, the attribute weight of the at least one target attribute is reallocated to obtain the first attribute weight corresponding to each target attribute; Based on the first attribute weight corresponding to each target attribute, the list data weight of each candidate list data is obtained respectively, and the valid list data is determined from the at least one candidate list data according to the list data weight; Each target attribute is configured with corresponding valid attribute conditions. The step of obtaining the list data weights for each candidate list based on the first attribute weights corresponding to each target attribute includes: For each candidate list data, based on the valid attribute conditions, the valid target attribute corresponding to the candidate list data is determined from the at least one target attribute, and the attributes other than the valid target attribute in the at least one target attribute are regarded as the invalid target attribute corresponding to the candidate list data. Based on the invalid target attributes, the first attribute weights corresponding to each valid target attribute are redistributed to obtain the second attribute weights of each valid target attribute corresponding to the candidate list data; The matching rate of each valid target attribute corresponding to the candidate list data is obtained, and the list data weight of the candidate list data is calculated based on the matching rate of each valid target attribute and the weight of the second attribute; the matching rate is used to characterize the degree of matching between the valid target attribute and the target screening condition; The list data weights of the candidate list data are calculated based on the matching rate of each corresponding valid target attribute and the weight of the second attribute, including: Multiply the matching rate of each valid target attribute by the weight of the second attribute to obtain the target matching rate of each valid target attribute; sum the target matching rates of each valid target attribute to obtain the list data weight of the candidate list data.
2. The list data filtering method according to claim 1, characterized in that, The method further includes: Set an initial weight for each of the preset attributes, and the sum of the initial weights of the preset attributes is 1.
3. The list data filtering method according to claim 2, characterized in that, The reassignment of attribute weights for the at least one target attribute based on the initial weights of the invalid attributes includes: The initial weights of the invalid attributes are summed to obtain the weight values to be assigned. The attribute weights of the at least one target attribute are reassigned based on the weight values to be assigned.
4. The list data filtering method according to claim 3, characterized in that, The step of reallocating the attribute weights of the at least one target attribute according to the unassigned weight values includes: Determine the proportional relationship between the attribute weights of the at least one target attribute, and reallocate the weights to be assigned to the attribute weights of the at least one target attribute according to the proportional relationship; the sum of the first attribute weights of the at least one target attribute after allocation is 1.
5. The list data filtering method according to claim 1, characterized in that, The step of determining valid list data from the at least one candidate list data according to the list data weight includes: Obtain the preset weight threshold for the list data; Candidate list data whose weight is greater than the weight threshold of the list data are considered as valid list data.
6. A list data filtering device, characterized in that, include: The list data screening module is used to determine at least one candidate list data from the list to be filtered based on the target screening conditions. An invalid attribute determination module is used to determine at least one target attribute corresponding to the target screening condition; Based on the at least one target attribute, invalid attributes that do not match the at least one target attribute are determined from a plurality of preset attributes; wherein each preset attribute is set with a corresponding initial weight; The weight reallocation module is used to reallocate the attribute weights of the at least one target attribute based on the initial weights of the invalid attribute, so as to obtain the first attribute weights corresponding to each target attribute. The list data determination module is used to obtain the list data weight of each candidate list data based on the first attribute weight corresponding to each target attribute, and determine the valid list data from the at least one candidate list data according to the list data weight; Each target attribute is configured with corresponding valid attribute conditions. The step of obtaining the list data weights for each candidate list based on the first attribute weights corresponding to each target attribute includes: For each candidate list data, based on the valid attribute conditions, the valid target attribute corresponding to the candidate list data is determined from the at least one target attribute, and the attributes other than the valid target attribute in the at least one target attribute are regarded as the invalid target attribute corresponding to the candidate list data. Based on the invalid target attributes, the first attribute weights corresponding to each valid target attribute are redistributed to obtain the second attribute weights of each valid target attribute corresponding to the candidate list data; The matching rate of each valid target attribute corresponding to the candidate list data is obtained, and the list data weight of the candidate list data is calculated based on the matching rate of each valid target attribute and the weight of the second attribute; the matching rate is used to characterize the degree of matching between the valid target attribute and the target screening condition; The list data weights of the candidate list data are calculated based on the matching rate of each corresponding valid target attribute and the weight of the second attribute, including: Multiply the matching rate of each valid target attribute by the weight of the second attribute to obtain the target matching rate of each valid target attribute; sum the target matching rates of each valid target attribute to obtain the list data weight of the candidate list data.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the list data filtering method according to any one of claims 1-5.