A big data blacklist filtering method

Abstract

The present invention relates to the field of big data technology, and in particular to a big data blacklist filtering method. The method comprises the following steps: acquiring blacklist number data from a preset database, thereby acquiring blacklist number data; generating blacklist sub-number data according to the blacklist number data, and classifying and marking the blacklist number data according to the blacklist sub-number data, thereby acquiring a blacklist number classification data set to send to a redis cache server; acquiring the number data to be detected, and extracting the classification mark of the number data to be detected, thereby acquiring the classification mark data of the number to be detected, so as to match the classification mark data of the number to be detected and the classification data set of the blacklist number, and obtain the classification mark matching result data. Compared with the traditional blacklist filtering method, the present invention can respond to user requests more quickly, and ensure the consistency and efficiency of the business process.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a big data blacklist filtering method. Background Art

[0002] Blacklist filtering is a technique or strategy used to identify and exclude specific types of data or entities. In the SMS industry, it is often necessary to filter some high-risk complaint mobile phone numbers before sending SMS. For example, before sending marketing SMS, in order to avoid the channel being shut down due to excessive complaint rates, these high-risk user mobile phone numbers need to be blocked. At present, conventional filtering methods are extremely inefficient in the face of massive data. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a big data blacklist filtering method to solve at least one of the above technical problems.

[0004] This application provides a big data blacklist filtering method, comprising the following steps:

[0005] Step S1: Obtain blacklist number data from a preset database to obtain blacklist number data;

[0006] Step S2: Generate blacklist sub-number data according to the blacklist number data, and classify and mark the blacklist number data according to the blacklist sub-number data, so as to obtain a blacklist number classification data set to send to the redis cache server;

[0007] Step S3: obtaining the data of the number to be detected, and extracting the classification mark of the number to be detected, thereby obtaining the classification mark data of the number to be detected, so as to match the classification mark data of the number to be detected with the classification data set of the blacklist number, and obtain the classification mark matching result data, wherein the classification mark matching result data includes matching success data and matching failure data;

[0008] Step S4: when it is determined that the classification mark matching result data is matching failure data, the number data to be detected is determined as whitelist detection number data;

[0009] Step S5: When it is determined that the classification mark matching result data is a successful match data, the number data to be detected and the blacklist number data corresponding to the blacklist number classification data set corresponding to the classification mark matching result data are matched to obtain the blacklist matching result data; if it is determined that the blacklist matching result data does not exist, the number data to be detected is determined as whitelist detection number data; if it is determined that the blacklist matching result data exists, the number data to be detected is determined as blacklist detection number data to perform a number filtering operation.

[0010] In the present invention, blacklist number data is obtained by a preset database, and its classification mark is sent to a redis cache server to form a blacklist classification data set. In this way, it can be ensured that blacklist numbers can be efficiently filtered out in real-time business scenarios, reducing the possibility of misjudgment and missed judgment. When the classification mark matching result data is a match failure, the number to be detected is determined as the whitelist detection number data, ensuring that the whitelist number will not be misjudged as a blacklist, and improving the accuracy of filtering. When the classification mark matching result data is a match success, the number to be detected is further matched with the corresponding blacklist number classification data set to obtain the blacklist matching result data, and the real blacklist number is clearly identified, thereby effectively filtering out potential risks. By utilizing the redis cache server, the reading speed and response efficiency of the data can be improved, thereby reducing the burden on the system and ensuring the efficient operation of the system. The method of the present invention has strong real-time performance when processing blacklist filtering, can quickly respond to user requests in real-time business scenarios, and ensures the consistency and efficiency of business processes.

[0011] Preferably, step S1 specifically includes:

[0012] Step S11: Obtain database request connection data;

[0013] Step S12: Connecting to the database according to the database request connection data, thereby obtaining database connection data;

[0014] Step S13: acquiring data according to the blacklist number collection requirement data in the database request connection data and the database connection data, thereby acquiring the blacklist number data.

[0015] In the present invention, through step S13, the required blacklist number data is efficiently obtained by using the preset database connection information and collection requirements. Through the preset database request connection, different databases can be connected according to specific needs, and flexible acquisition of different data sources is achieved. Through the database connection, the latest blacklist data can be obtained in time, ensuring the real-time nature of the data. Through automated database connection and data acquisition, the workload of manual operations is reduced and work efficiency is improved. Through the preset database connection, the legitimacy and security of the source of the data can be ensured.

[0016] Preferably, step S2 specifically includes:

[0017] Step S21: extracting the number digit data of the blacklist number data by using the preset sub-number threshold data, thereby obtaining the blacklist sub-number data;

[0018] Step S22: Classify and mark the blacklist number data according to the blacklist sub-number data, thereby obtaining a blacklist number classification data set to send to the redis cache server.

[0019] In the present invention, by step S21, bit data extraction is performed according to a preset sub-number threshold value, so that the classification mark of the blacklist number is more refined, and the accuracy of the classification is improved. By sending the blacklist number classification data set to the redis cache server (step S22), the reading speed and response efficiency of the data can be improved, and the efficient operation of the system is guaranteed. Since only the extracted sub-number data is stored, the consumption of storage space can be reduced compared to storing complete blacklist number data. By extracting the sub-number data, it is possible to avoid directly exposing the complete number information while ensuring the classification of the blacklist number, thereby protecting the privacy information.

[0020] Preferably, step S3 specifically includes:

[0021] Step S31: obtaining the data of the number to be detected, and extracting the number position data of the number to be detected through the preset sub-number threshold data, thereby obtaining the classification mark data of the number to be detected;

[0022] Step S32: matching is performed according to the classification mark data of the number to be detected and the classification data set of the blacklist number to obtain classification mark matching result data, wherein the classification mark matching result data includes matching success data and matching failure data.

[0023] In the present invention, through step S31, the data of the number to be detected is classified and marked, and the data to be matched is prepared in advance, so that the matching process is more efficient. Through step S32, the matching results are clarified, including the data of successful matching and the data of failed matching, which provides a clear basis for subsequent processing. Classifying and marking the numbers to be detected and extracting data in advance can reduce the time of real-time matching and improve the speed and efficiency of matching. By pre-processing the numbers to be detected, the consumption of system resources in the real-time matching process can be reduced, and the performance of the system can be improved.

[0024] Preferably, step S5 is specifically:

[0025] Step S51: when it is determined that the classification mark matching result data is the matching success data, multi-dimensional matching is performed on the blacklist number data corresponding to the blacklist number classification data set corresponding to the classification mark matching result data, so as to obtain the blacklist matching result data, wherein the multi-dimensional matching includes historical record matching and number matching;

[0026] Step S52: if it is determined that the blacklist matching result data does not exist, the number data to be detected is determined as whitelist detection number data;

[0027] Step S53: If it is determined that the blacklist matching result data is matching result data, the to-be-detected number data is determined as blacklist detection number data to perform a number filtering operation.

[0028] In the present invention, through step S51, multi-dimensional matching including historical record matching and number matching is adopted to improve the accuracy and comprehensiveness of matching. Step S53 determines the number to be detected as blacklist detection number data, ensures that the number identified as blacklist will be filtered out, and improves the security of the system. Step S52 determines the number to be detected as whitelist detection number data, ensures that the whitelist number will not be misjudged as blacklist, and improves the accuracy of filtering. Through accurate classification marking and matching, the consumption of system resources can be reduced, and the overall performance of the system is improved.

[0029] Preferably, step S12 is specifically:

[0030] Step S121: obtaining system current status data, wherein the system current status data includes system load data, database server performance data, and connection request frequency data;

[0031] Step S122: Select a connection pool strategy based on the current system status data, thereby obtaining connection pool strategy data;

[0032] Step S123: Create a connection pool according to the connection pool policy data, thereby obtaining real-time connection pool creation data;

[0033] Step S124: Create data in the real-time connection pool to connect to the database, thereby obtaining database connection data.

[0034] In the present invention, through steps S121 and S122, a connection pool strategy is dynamically selected according to the system status data obtained in real time, so that connection management is more flexible and efficient. Step S122 ensures that the selected connection pool strategy is optimal through a preset strategy selection algorithm, which is conducive to improving the efficiency of database connection. Through step S123, a connection pool is created in real time according to the connection pool strategy data, which ensures the real-time nature of the connection pool and conforms to the current system state. Through step S124, a database connection is established using the connection pool created in real time, which improves the efficiency of the database connection and ensures the efficient operation of the system.

[0035] Preferably, step S21 is specifically:

[0036] Step S211: extracting the number digit data of the blacklist number data by using the preset first sub-number threshold data, thereby obtaining the first preliminary blacklist sub-number data;

[0037] Step S212: extracting the number digit data of the blacklist number data by using the preset second sub-number threshold data, thereby obtaining the second preliminary blacklist sub-number data;

[0038] Step S213: performing number information entropy calculation on the first preliminary blacklist sub-number data and the second preliminary blacklist sub-number data, thereby obtaining first sub-number information entropy data and second sub-number information entropy data respectively; wherein the number information entropy calculation is performed by using a number information entropy calculation formula, and the number information entropy calculation formula is specifically:

[0039]

[0040] H(X) is the sub-number information entropy data, i is the number sequence item, N is the number of different numbers in the blacklist sub-number data, P(x i ) is the probability data of the i-th number appearing, x i is the ith sub-number data of the blacklist sub-number data, α is the weight coefficient of the probability data of the ith number appearing, β is the weight coefficient of the territorial data of the ith number, L(x i ) is the location data of the i-th number, γ is the weight coefficient of the length data of the i-th number, Len(x i ) is the length data of the i-th number, δ is the weight coefficient of the characteristic data of the i-th number, F(x i ) is the characteristic data of the ith number, and u is the error adjustment item of the sub-number information entropy data;

[0041] Step S214: when it is determined that the first sub-number information entropy data is greater than or equal to the second sub-number information entropy data, the first sub-number information entropy data is determined as blacklist sub-number data;

[0042] Step S215: When it is determined that the first sub-number information entropy data is less than the second sub-number information entropy data, the second sub-number information entropy data is determined as blacklist sub-number data.

[0043] The present invention constructs a number information entropy calculation formula, which evaluates the importance of each number, thereby providing a preliminary blacklist sub-number data corresponding to the sub-number information entropy for comparison, so as to improve the accuracy of classification and avoid the problem of excessive system overhead caused by number similarity. Wherein H(X) is the sub-number information entropy data, which represents the degree of confusion or randomness of the sub-number data. i is the number sequence item, which represents the sequence numbers of different numbers in the blacklist sub-number data. N is the number of different numbers in the blacklist sub-number data, which is used to calculate the probability of each number appearing. P(x i) is the probability data of the occurrence of the ith number, which indicates the relative frequency of the occurrence of the ith number in the entire data set. The parameters in the formula can be dynamically adjusted according to the actual situation, and the appropriate weight coefficient can be determined according to the distribution of experimental or actual data, thereby improving the flexibility and adaptability of the calculation. Conventional technical means often only divide blacklist numbers based on a single indicator, and cannot avoid the system overhead problem caused by a single division method for number concentration. This sub-number classification method based on information entropy calculation can classify blacklist numbers more accurately and improve filtering accuracy.

[0044] In the present invention, by steps S211 and S212, bit data is extracted according to a preset sub-number threshold, so that the sub-number extraction of the blacklist number is more refined, and the accuracy of classification is improved. Step S213 uses information entropy calculation to intelligently classify the sub-numbers, reduce the load problem caused by excessive concentration of similar data caused by only single-bit data division, and ensure the accuracy of classification. By comparing the size of information entropy, a multi-dimensional classification strategy is implemented, and the comprehensiveness and accuracy of classification are improved. Through the intelligent classification strategy, the misjudgment of normal numbers is reduced, and the accuracy of filtering is improved.

[0045] Preferably, step S22 is specifically:

[0046] Step S221: classify and mark the blacklist number data according to the blacklist sub-number data, thereby obtaining a preliminary blacklist number classification data set;

[0047] Step S222: performing location marking on the preliminary blacklist number classification data in the preliminary blacklist number classification data set, thereby obtaining a location marked number classification data set;

[0048] Step S223: Perform maximum error clustering calculation on the location mark number classification data in the location mark number classification data set, so as to obtain a blacklist number classification data set to send to the redis cache server.

[0049] In the present invention, the blacklist numbers are classified and marked by step S221, so that the classification is more refined and the accuracy of the classification is improved. Step S222 enriches the characteristic information of the number by adding territorial information, which helps to cluster more accurately. Step S223 clusters similar numbers into the same category through maximum error clustering calculation, thereby improving the comprehensiveness and accuracy of the classification. The final blacklist number classification data set is sent to the redis cache server, which realizes fast storage and query and improves the response speed of the system.

[0050] Preferably, step S223 is specifically:

[0051] Step S2231: generating classification quantity data according to the territorial mark number classification data in the territorial mark number classification data set;

[0052] Step S2232: extracting number length features and number change features from the territorial mark number classification data, thereby obtaining number length feature data and number change feature data;

[0053] Step S2233: Perform cluster calculation on the number length feature data and the number change feature data according to the classification quantity data, so as to obtain a blacklist number classification data set to send to the redis cache server.

[0054] In the present invention, through step S2232, comprehensive features including number length features and number change features are extracted from the location-marked number classification data, enriching the description and classification basis of the number. Step S2233 uses the classification quantity data to perform cluster calculation on the feature data, clustering similar numbers together, and improving the accuracy and comprehensiveness of the classification. The final blacklist number classification data set is sent to the redis cache server, which realizes fast storage and query and improves the response speed of the system. By introducing different features and combining the clustering method, the classification is made more accurate and refined, and the misjudgment rate is reduced.

[0055] Preferably, the classification quantity data generated in step S2231 is calculated by a number characteristic classification quantity calculation formula, wherein the number characteristic classification quantity calculation formula is specifically:

[0056]

[0057] Q is the number of classification data, θ is the weight coefficient of the number of classification data of the territorial mark number, b is the constant term of the territorial mark base, k is the number of classification data of the territorial mark number, δ is the weight coefficient of the number of types of classification data of the territorial mark number, m is the number of types of classification data of the territorial mark number, x is the classification data of the territorial mark number, f(x) is the mapping item of the territorial mark number characteristics, p i Probability distribution data for territorial mark number classification data.

[0058] The present invention constructs a number characteristic classification quantity calculation formula. The classification quantity data provided by the formula is further clustered to optimize the classification results of blacklist numbers and improve the filtering accuracy. The present invention makes dynamic adjustments according to actual conditions and can determine appropriate weight coefficients according to the distribution of experimental or actual data, thereby improving the flexibility and adaptability of calculation. The introduced characteristic mapping term f(x) makes the calculation process more comprehensive and more sensitive to characteristic changes. Where p iIt is the probability distribution data of the territorial mark number classification data, which represents the probability distribution of each classification. It is usually represented by the frequency of data occurrence / total number of data, and can also be replaced by a probability distribution function. The weight coefficients (θ, δ) of different parameters can be used to balance the influence of quantity data and type quantity data, making the evaluation more reasonable and comprehensive. Conventional technical means are limited by their dependence on specific characteristics or insufficient trade-offs, while the present invention can more comprehensively reflect the characteristics of the number of classifications through dynamic adjustment of parameters, thereby improving the classification accuracy and enhancing the effect of blacklist filtering.

[0059] Preferably, step 51 specifically includes:

[0060] Step 511: when it is determined that the classification mark matching result data is the matching success data, the number data to be detected and the blacklist number data corresponding to the blacklist number classification data set corresponding to the classification mark matching result data are matched to obtain the number matching result data;

[0061] Step 512: When it is determined that the number matching result data is number matching success result data, the number matching result data is determined as blacklist matching result data;

[0062] Step 513: when it is determined that the number matching result data is number matching failure result data, historical data acquisition is performed according to the number data to be detected, thereby acquiring historical call data;

[0063] Step 514: Match the preset call behavior parameters according to the call record data and the interaction record data of the historical call data, so as to obtain blacklist matching result data.

[0064] The present invention introduces a multi-dimensional matching strategy, which not only takes into account the direct matching of numbers, but also takes into account the historical call records, thereby improving the accuracy of matching. Step S513 makes full use of the historical call data of the number to be detected, provides more information for matching, expands the role of blacklist screening in the actual process, and increases the comprehensiveness of matching. Step S514 introduces preset call behavior parameters, which can be flexibly adjusted according to actual needs, making the matching more targeted. By comprehensively considering information in multiple dimensions, the accuracy of blacklist matching is improved and the misjudgment rate is reduced.

[0065] The beneficial effects of the present invention are: blacklist number data is quickly obtained through a preset database, and a Redis cache server is used for efficient classification and marking. This ensures the real-time and efficiency of the whole process. The method allows the classification and marking rules to be flexibly preset and adjusted according to specific needs to adapt to blacklist filtering in different scenarios and needs. The number to be detected is matched with the blacklist classification data set, and the matching result is classified and marked, so as to accurately distinguish the data of successful matching and failed matching. When the classification mark matching result data is a match failure, the number to be detected is determined as whitelist detection number data, so as to avoid misjudgment of legitimate numbers. For the data that successfully matches, multi-dimensional matching is performed, including historical record matching and number matching, so as to improve the filtering accuracy and reliability. For the data that failed to match, by obtaining historical call data, matching is performed in combination with preset call behavior parameters to determine whether it is blacklist detection number data. It is suitable for processing large-scale blacklist number data, and the scalability and applicability of the method are guaranteed. Through multi-dimensional matching and historical data analysis, the possibility of misjudgment is reduced, the accuracy of filtering is improved, and the communication security of users is protected. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting implementations made with reference to the following drawings:

[0067] Figure 1 A flowchart showing the steps of a big data blacklist filtering method according to an embodiment is shown;

[0068] Figure 2 A flowchart of step S1 of an embodiment is shown;

[0069] Figure 3 A flowchart of step S5 of an embodiment is shown;

[0070] Figure 4 A flowchart of step S12 of an embodiment is shown;

[0071] Figure 5 A step flow chart of step S21 of an embodiment is shown. DETAILED DESCRIPTION

[0072] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0073] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0074] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0075] See also Figures 1 to 5 , this application provides a big data blacklist filtering method, comprising the following steps:

[0076] Step S1: Obtain blacklist number data from a preset database to obtain blacklist number data;

[0077] Specifically, for example, a query is made using the structured query language (SQL), such as: SELECT phone_number; FROM black_list_database; WHERE criteria_condition = 'X'; here, black_list_database is a preset database table, and criteria_condition is a condition, through which the required blacklist number data is screened out.

[0078] Step S2: Generate blacklist sub-number data according to the blacklist number data, and classify and mark the blacklist number data according to the blacklist sub-number data, so as to obtain a blacklist number classification data set to send to the redis cache server;

[0079] Specifically, for example, the blacklist number data is marked with a fixed number of digits to obtain the key value - blacklist number data, which has been sent to the redis cache server.

[0080] Specifically, for example, here, black_list_database is a preset database table, and criteria_condition is a condition, through which the required blacklist number data is filtered out. import redis # Connect to the Redis server redis_client = redis.StrictRedis (host = 'localhost', port = 6379, decode_responses = True); # Classify and mark the blacklist numbers for phone_number in black_list_data: category = classify (phone_number) # Custom classification function redis_client.sadd (category, phone_number).

[0081] Step S3: obtaining the data of the number to be detected, and extracting the classification mark of the number to be detected, thereby obtaining the classification mark data of the number to be detected, so as to match the classification mark data of the number to be detected with the classification data set of the blacklist number, and obtain the classification mark matching result data, wherein the classification mark matching result data includes matching success data and matching failure data;

[0082] Specifically, for example, in an actual application, the number data to be detected can be obtained through the interface, and then classified and marked, and matched with the classified data in Redis. This involves specific business logic, such as classification by number structure or other rules.

[0083] Step S4: when it is determined that the classification mark matching result data is matching failure data, the number data to be detected is determined as whitelist detection number data;

[0084] Specifically, for example, if the match fails, the system will mark the number to be detected as a whitelist number for subsequent processing.

[0085] Step S5: When it is determined that the classification mark matching result data is a successful match data, the number data to be detected and the blacklist number data corresponding to the blacklist number classification data set corresponding to the classification mark matching result data are matched to obtain the blacklist matching result data; if it is determined that the blacklist matching result data does not exist, the number data to be detected is determined as whitelist detection number data; if it is determined that the blacklist matching result data exists, the number data to be detected is determined as blacklist detection number data to perform a number filtering operation.

[0086] Specifically, for example, if the match is successful, the system will match the number to be detected with the corresponding blacklist data set to obtain blacklist matching result data.

[0087] Specifically, for example, running on a redis cache server, storing massive blacklist numbers in the redis cache server: 101: querying massive blacklists from the database, 102: using the last 3 digits of the mobile phone number as the key, and the mobile phone numbers with the same last 3 digits as the value (set), 103: storing in the form of key-value pairs in the redis cache server. Filtering blacklist numbers: 201: receiving a new number request, 202: intercepting the last 3 digits of the new number, 203: searching for the redis server key based on the last 3 digits of the new number, 204: if it does not exist, it means that this number is not on the blacklist, 205: if it exists, continue to take out the corresponding value (set), and search one by one from the value to see if there is a match, 206: if there is a match, it means that this number is on the blacklist, and the number is filtered out immediately. In massive blacklist filtering, the last 3 digits of the blacklist number are stored as the key in the cache server. When comparing, the last 3 digits of the number are first searched in the cache server. If it exists, it is compared one by one. The total number of comparisons is reduced by 1,000 times compared with the traditional method.

[0088] In the present invention, blacklist number data is obtained by a preset database, and its classification mark is sent to a redis cache server to form a blacklist classification data set. In this way, it can be ensured that blacklist numbers can be efficiently filtered out in real-time business scenarios, reducing the possibility of misjudgment and missed judgment. When the classification mark matching result data is a match failure, the number to be detected is determined as the whitelist detection number data, ensuring that the whitelist number will not be misjudged as a blacklist, and improving the accuracy of filtering. When the classification mark matching result data is a match success, the number to be detected is further matched with the corresponding blacklist number classification data set to obtain the blacklist matching result data, and the real blacklist number is clearly identified, thereby effectively filtering out potential risks. By utilizing the redis cache server, the reading speed and response efficiency of the data can be improved, thereby reducing the burden on the system and ensuring the efficient operation of the system. The method of the present invention has strong real-time performance when processing blacklist filtering, can quickly respond to user requests in real-time business scenarios, and ensures the consistency and efficiency of business processes.

[0089] Preferably, step S1 specifically includes:

[0090] Step S11: Obtain database request connection data;

[0091] Specifically, for example, a web application collects database connection data by filling out a form to provide the following information: database type (such as MySQL, PostgreSQL, etc.), host address, port number, user name, password, database name, blacklist number collection requirement data (optional).

[0092] Step S12: Connecting to the database according to the database request connection data, thereby obtaining database connection data;

[0093] Specifically, for example, in Python, you can use the pymysql library to connect to the MySQL database: import pymysql # Use the connection information provided by the user to connect to the database conn = pymysql.connect(host = host, port = port, user = user, password = password, db = database) # Get a cursor object cursor = conn.cursor() # Execute query operations or other database operations #... # Close the connection conn.close().

[0094] Step S13: acquiring data according to the blacklist number collection requirement data in the database request connection data and the database connection data, thereby acquiring the blacklist number data.

[0095] Specifically, for example, use the data and database connection provided by the user to execute a query: #Assume that the query statement provided by the user is: #SELECT phone_number FROM black_list WHERE criteria_condition='X'; #Execute the query cursor.execute(user_provided_query) #Get the query result black_list_data=cursor.fetchall() #Close the connection conn.close().

[0096] In the present invention, through step S13, the required blacklist number data is efficiently obtained by using the preset database connection information and collection requirements. Through the preset database request connection, different databases can be connected according to specific needs, and flexible acquisition of different data sources is achieved. Through the database connection, the latest blacklist data can be obtained in time, ensuring the real-time nature of the data. Through automated database connection and data acquisition, the workload of manual operations is reduced and work efficiency is improved. Through the preset database connection, the legitimacy and security of the source of the data can be ensured.

[0097] Preferably, step S2 specifically includes:

[0098] Step S21: extracting the number digit data of the blacklist number data by using the preset sub-number threshold data, thereby obtaining the blacklist sub-number data;

[0099] Specifically, for example, a sub-number threshold is preset to 3, indicating that the last 3 digits of the blacklist number need to be extracted as the sub-number data. Implementation in Python: def extract_subnumber(phone_number,threshold):sub_number=phone_number[-threshold:]return sub_number#Exampleblack_list_number='1234567890',threshold=3,sub_number=extract_subnumber(black_list_number,threshold),print(sub_number); If there is a blacklist number '1234567890', the system extracts the last 3 digits, and the result is '890'.

[0100] Step S22: Classify and mark the blacklist number data according to the blacklist sub-number data, thereby obtaining a blacklist number classification data set to send to the redis cache server.

[0101] Specifically, for example, use Python to connect to the Redis server and store the classified data in Redis: import redis def classify_and_send_to_redis(sub_number,category): #Connect to the Redis server redis_client = redis.StrictRedis(host = 'localhost',port = 6379,decode_responses = True) #Classification mark classified_data = category + ':' + sub_number #Store in Redis; redis_client.sadd(category,sub_number) #Close the connection redis_client.close() #Example sub_number = '890', category = 'area_code', classify_and_send_to_redis(sub_number,category); Store the blacklist sub-number data ('890') in the Redis cache server according to the preset classification mark (for example, 'area_code').

[0102] Specifically, for example, the last 3 digits of a mobile phone number are used as the key, and the mobile phone numbers with the same last 3 digits are used as the value (set), which are stored in the redis cache server in the form of a key-value pair.

[0103] In the present invention, by step S21, bit data extraction is performed according to a preset sub-number threshold value, so that the classification mark of the blacklist number is more refined, and the accuracy of the classification is improved. By sending the blacklist number classification data set to the redis cache server (step S22), the reading speed and response efficiency of the data can be improved, and the efficient operation of the system is guaranteed. Since only the extracted sub-number data is stored, the consumption of storage space can be reduced compared to storing complete blacklist number data. By extracting the sub-number data, it is possible to avoid directly exposing the complete number information while ensuring the classification of the blacklist number, thereby protecting the privacy information.

[0104] Preferably, step S3 specifically includes:

[0105] Step S31: obtaining the data of the number to be detected, and extracting the number position data of the number to be detected through the preset sub-number threshold data, thereby obtaining the classification mark data of the number to be detected;

[0106] Specifically, for example, there is a number to be detected, and the preset sub-number threshold is 3, which means that the last 3 digits of the number to be detected are extracted as sub-number data. For example, if there is a number to be detected as '9876543210', the system extracts the last 3 digits, and the result is '210'.

[0107] Step S32: matching is performed according to the classification mark data of the number to be detected and the classification data set of the blacklist number to obtain classification mark matching result data, wherein the classification mark matching result data includes matching success data and matching failure data.

[0108] Specifically, for example, after connecting to the Redis server, check whether the classification mark data of the number to be detected matches the data in the blacklist number classification data set, and return True if the match is successful, otherwise return False.

[0109] In the present invention, through step S31, the data of the number to be detected is classified and marked, and the data to be matched is prepared in advance, so that the matching process is more efficient. Through step S32, the matching results are clarified, including the data of successful matching and the data of failed matching, which provides a clear basis for subsequent processing. Classifying and marking the numbers to be detected and extracting data in advance can reduce the time of real-time matching and improve the speed and efficiency of matching. By pre-processing the numbers to be detected, the consumption of system resources in the real-time matching process can be reduced, and the performance of the system can be improved.

[0110] Preferably, step S5 is specifically:

[0111] Step S51: when it is determined that the classification mark matching result data is the matching success data, multi-dimensional matching is performed on the blacklist number data corresponding to the blacklist number classification data set corresponding to the classification mark matching result data, so as to obtain the blacklist matching result data, wherein the multi-dimensional matching includes historical record matching and number matching;

[0112] Specifically, for example, it is necessary to integrate historical record matching and number matching. Assuming that the system has obtained historical record data and number data, use Python for multi-dimensional matching: def multi_dimensional_matching(history_data, phone_data): #Assuming that some algorithms are used for historical record matching and number matching, this is just an example matched_data = history_matching_algorithm(history_data) + phone_matching_algorithm(phone_data) return matched_data #Example history_data = [...] #Assuming it is a list of historical record data phone_data = [...] #Assuming it is a list of number data result = multi_dimensional_matching(history_data, phone_data), print(result), assuming that history_matching_algorithm and phone_matching_algorithm are actual matching algorithms. By integrating the two matching results, the result of multi-dimensional matching is obtained.

[0113] Specifically, for example, historical record matching is to determine whether to add to the blacklist based on the historical records of the number to be detected. Specific rules include but are not limited to making no less than 200 calls a day, no less than 50 calls are completed within 10 seconds, and the duration is 2 weeks.

[0114] Step S52: if it is determined that the blacklist matching result data does not exist, the number data to be detected is determined as whitelist detection number data;

[0115] Specifically, for example, if the matching result is not present, the number to be detected will be determined as a whitelist detection number.

[0116] Step S53: If it is determined that the blacklist matching result data is matching result data, the to-be-detected number data is determined as blacklist detection number data to perform a number filtering operation.

[0117] Specifically, for example, if the matching result is existence, then the number to be detected will be determined as a blacklist detection number for filtering operations.

[0118] In the present invention, through step S51, multi-dimensional matching including historical record matching and number matching is adopted to improve the accuracy and comprehensiveness of matching. Step S53 determines the number to be detected as blacklist detection number data, ensures that the number identified as blacklist will be filtered out, and improves the security of the system. Step S52 determines the number to be detected as whitelist detection number data, ensures that the whitelist number will not be misjudged as blacklist, and improves the accuracy of filtering. Through accurate classification marking and matching, the consumption of system resources can be reduced, and the overall performance of the system is improved.

[0119] Preferably, step S12 is specifically:

[0120] Step S121: obtaining system current status data, wherein the system current status data includes system load data, database server performance data, and connection request frequency data;

[0121] Specifically, for example, in actual situations, system monitoring tools such as Zabbix, Prometheus, etc. are used to obtain the current status data of the system. The specific acquisition method will vary depending on the monitoring tool used. For example, use the psutil library in Python to obtain the current status data of the system: import psutil # Get system load data cpu_load = psutil.cpu_percent() # Get database server performance data # Here, it is assumed that memory usage and disk usage are obtained memory_usage = psutil.virtual_memory().percent; disk_usage = psutil.disk_usage(' / ').percent; # Get connection request frequency data # Here, it is assumed that the request frequency is obtained from the network layer network_requests = get_network_requests(); # Return the obtained data system_status_data = {"cpu_load":cpu_load,"memory_usage":memory_usage,"disk_usage":disk_usage,"network_requests":network_requests}.

[0122] Step S122: Select a connection pool strategy based on the current system status data, thereby obtaining connection pool strategy data;

[0123] Specifically, for example, def select_connection_pool_strategy(system_status_data): #Assume that the strategy selection rule here is to select based on CPU load and memory usage if system_status_data["cpu_load"]<50andsystem_status_data["memory_usage"]<70:return "Strategy A" else:return "Strategy B".

[0124] Specifically, for example, the connection pool strategy: Maximum number of connections: Set the maximum number of connections to 100, that is, a maximum of 100 database connections can be maintained in the connection pool at the same time. Minimum number of idle connections: Set the minimum number of idle connections to 10, that is, the minimum number of idle connections maintained in the connection pool is 10. Connection timeout: Set the connection timeout to 30 seconds, that is, if a connection is not used within 30 seconds, it will be considered a timed-out connection. Maximum waiting time: Set the maximum waiting time to 5 seconds, that is, if all connections in the connection pool are occupied, a request will wait for a maximum of 5 seconds. If this time is exceeded, a connection timeout exception will be thrown.

[0125] Step S123: Create a connection pool according to the connection pool policy data, thereby obtaining real-time connection pool creation data;

[0126] Specifically, for example, a corresponding connection pool is created according to the selected strategy. For example, def create_connection_pool(strategy): if strategy == "StrategyA": #Create a connection pool corresponding to StrategyA connection_pool = create_pool_for_strategy_A() elif strategy == "StrategyB": #Create a connection pool corresponding to StrategyB connection_pool = create_pool_for_strategy_B() return connection_pool.

[0127] Specifically, for example, when the system starts, according to the policy, 10 initialization connections are created and placed in the connection pool. When the system is running, whenever a business request needs to use a database connection, the connection pool will first check whether there is an idle connection. If there is, it will be directly allocated to the request. If all the connections in the connection pool have been used, but the maximum number of connections has not been exceeded, the connection pool will create a new connection according to the policy and allocate it to the request. If all the connections in the connection pool have been used and the maximum number of connections has been reached, the connection pool will wait for a connection to be released according to the maximum waiting time set by the policy. If a connection exceeds the connection timeout, it will be closed and removed from the connection pool.

[0128] Step S124: Create data in the real-time connection pool to connect to the database, thereby obtaining database connection data.

[0129] Specifically, for example, the database connection is performed according to the connection pool created in the previous step, such as def connect_to_database(connection_pool):connection=connection_pool.get_connection()returnconnection.

[0130] In the present invention, through steps S121 and S122, a connection pool strategy is dynamically selected according to the system status data obtained in real time, so that connection management is more flexible and efficient. Step S122 ensures that the selected connection pool strategy is optimal through a preset strategy selection algorithm, which is conducive to improving the efficiency of database connection. Through step S123, a connection pool is created in real time according to the connection pool strategy data, which ensures the real-time nature of the connection pool and conforms to the current system state. Through step S124, a database connection is established using the connection pool created in real time, which improves the efficiency of the database connection and ensures the efficient operation of the system.

[0131] Preferably, step S21 is specifically:

[0132] Step S211: extracting the number digit data of the blacklist number data by using the preset first sub-number threshold data, thereby obtaining the first preliminary blacklist sub-number data;

[0133] Specifically, for example, the first sub-number threshold is 3, which means extracting the first (last) three digits of the blacklist number as the first preliminary blacklist sub-number data.

[0134] Step S212: extracting the number digit data of the blacklist number data by using the preset second sub-number threshold data, thereby obtaining the second preliminary blacklist sub-number data;

[0135] Specifically, for example, the second sub-number threshold is 7, which means that the first (last) seven digits of the blacklist number are extracted as the second preliminary blacklist sub-number data.

[0136] Step S213: performing number information entropy calculation on the first preliminary blacklist sub-number data and the second preliminary blacklist sub-number data, thereby obtaining first sub-number information entropy data and second sub-number information entropy data respectively;

[0137] Specifically, for example, #specific values, the first sub-number information entropy data: alpha_1=0.3, beta_1=0.5, gamma_1=0.2, delta_1=0.4, u_1=0.1, the second sub-number information entropy data: alpha_2=0.4, beta_2=0.6, gamma_2=0.3, delta_2=0.2, u_2=0.15. #Calculate information entropy according to the formula def calculate_entropy(subnumber,alpha,beta,gamma,delta,u): #Assume that various data can be obtained here P_xi=get_P_xi(subnumber), L_xi=get_L_xi(subnumber), Len_xi=get_Len_xi(subnumber), F_xi=get_F_xi(subnumber), entropy=-sum(P_xi*((alpha*log2(P_xi)+beta*log2(L_xi)) / (gamma*log2(Len_xi)+delta*log2(F_xi)))for P_xi,L_xi,Len_xi,F_xiin zip(P_xi,L_xi,Len_xi,F_xi))+u,return entropy.

[0138] Specifically, for example, the basic information entropy formula alog2x+blog2y is used for calculation, wherein a is the first weight, x is the first preliminary blacklist sub-number data, b is the second weight, and y is the second preliminary blacklist sub-number data.

[0139] The number information entropy is calculated by the number information entropy calculation formula, and the number information entropy calculation formula is specifically as follows:

[0140]

[0141] H(X) is the sub-number information entropy data, i is the number sequence item, N is the number of different numbers in the blacklist sub-number data, P(x i ) is the probability data of the i-th number appearing, x iis the ith sub-number data of the blacklist sub-number data, α is the weight coefficient of the probability data of the ith number appearing, β is the weight coefficient of the territorial data of the ith number, L(x i ) is the location data of the i-th number, γ is the weight coefficient of the length data of the i-th number, Len(x i ) is the length data of the i-th number, δ is the weight coefficient of the characteristic data of the i-th number, F(x i ) is the characteristic data of the ith number, and u is the error adjustment item of the sub-number information entropy data;

[0142] Step S214: when it is determined that the first sub-number information entropy data is greater than or equal to the second sub-number information entropy data, the first sub-number information entropy data is determined as blacklist sub-number data;

[0143] Specifically, for example, assuming that the first sub-number information entropy data is 3.2 and the second sub-number information entropy data is 2.8, then the first sub-number information entropy data is greater than the second sub-number information entropy data, and therefore the first sub-number information entropy data is determined as blacklist sub-number data.

[0144] Step S215: When it is determined that the first sub-number information entropy data is less than the second sub-number information entropy data, the second sub-number information entropy data is determined as blacklist sub-number data.

[0145] Specifically, for example, assuming that the first sub-number information entropy data is 3.5 and the second sub-number information entropy data is 4.8, then the first sub-number information entropy data is less than the second sub-number information entropy data, and therefore the second sub-number information entropy data is determined as blacklist sub-number data.

[0146] The present invention constructs a number information entropy calculation formula, which evaluates the importance of each number, thereby providing a preliminary blacklist sub-number data corresponding to the sub-number information entropy for comparison, so as to improve the accuracy of classification and avoid the problem of excessive system overhead caused by number similarity. Wherein H(X) is the sub-number information entropy data, which represents the degree of confusion or randomness of the sub-number data. i is the number sequence item, which represents the sequence numbers of different numbers in the blacklist sub-number data. N is the number of different numbers in the blacklist sub-number data, which is used to calculate the probability of each number appearing. P(x i) is the probability data of the occurrence of the ith number, which indicates the relative frequency of the occurrence of the ith number in the entire data set. The parameters in the formula can be dynamically adjusted according to the actual situation, and the appropriate weight coefficient can be determined according to the distribution of experimental or actual data, thereby improving the flexibility and adaptability of the calculation. Conventional technical means often only divide blacklist numbers based on a single indicator, and cannot avoid the system overhead problem caused by a single division method for number concentration. This sub-number classification method based on information entropy calculation can classify blacklist numbers more accurately and improve filtering accuracy.

[0147] In the present invention, by steps S211 and S212, bit data is extracted according to a preset sub-number threshold, so that the sub-number extraction of the blacklist number is more refined, and the accuracy of classification is improved. Step S213 uses information entropy calculation to intelligently classify the sub-numbers, reduce the load problem caused by excessive concentration of similar data caused by only single-bit data division, and ensure the accuracy of classification. By comparing the size of information entropy, a multi-dimensional classification strategy is implemented, and the comprehensiveness and accuracy of classification are improved. Through the intelligent classification strategy, the misjudgment of normal numbers is reduced, and the accuracy of filtering is improved.

[0148] Preferably, step S22 is specifically:

[0149] Step S221: classify and mark the blacklist number data according to the blacklist sub-number data, thereby obtaining a preliminary blacklist number classification data set;

[0150] Specifically, for example, there is a classification model that can classify blacklisted numbers, and the model outputs labels such as "Spam" (junk information) or "Fraud" (fraudulent information).

[0151] Step S222: performing location marking on the preliminary blacklist number classification data in the preliminary blacklist number classification data set, thereby obtaining a location marked number classification data set;

[0152] Specifically, for example, the location mark can be obtained by querying the number location database.

[0153] Step S223: Perform maximum error clustering calculation on the location mark number classification data in the location mark number classification data set, so as to obtain a blacklist number classification data set to send to the redis cache server.

[0154] Specifically, for example, there is a territorial mark number classification data set, which includes the following territorial marks and their corresponding number of numbers: territorial mark A: [10, 12, 9, 11, 10], territorial mark B: [8, 7, 9, 6, 7], territorial mark C: [15, 14, 16, 17, 15], calculate the average value of the number under each territorial mark: the average value of territorial mark A is 10.4, the average value of territorial mark B is 7.4, and the average value of territorial mark C is 15.4. Next, calculate the maximum error between the number and the average value under each territorial mark: the error of territorial mark A is [0.4, 1.6, 1.4, 0.6, 0.4], the error of territorial mark B is [0.6, 0.4, 1.6, 0.4, 0.6], and the error of territorial mark C is [0.6, 1.6, 0.6, 1.6, 0.6]. Then, the errors are clustered. Assume that a clustering threshold is set to 1, which means that numbers with an error of less than or equal to 1 will be classified into the same category. After clustering calculation, the system obtains the following clustering results: Clustering results of location mark A: [[10,11],

[12] ,[9]], clustering results of location mark B: [[8,7,9,6,7]], clustering results of location mark C: [[15,14,16,17,15]], the system uses these clustering results as blacklist number classification data sets and sends them to the redis cache server for subsequent use.

[0155] In the present invention, the blacklist numbers are classified and marked by step S221, so that the classification is more refined and the accuracy of the classification is improved. Step S222 enriches the characteristic information of the number by adding territorial information, which helps to cluster more accurately. Step S223 clusters similar numbers into the same category through maximum error clustering calculation, thereby improving the comprehensiveness and accuracy of the classification. The final blacklist number classification data set is sent to the redis cache server, which realizes fast storage and query and improves the response speed of the system.

[0156] Preferably, step S223 is specifically:

[0157] Step S2231: generating classification quantity data according to the territorial mark number classification data in the territorial mark number classification data set;

[0158] Specifically, for example, the number of numbers under each territory mark is counted.

[0159] Step S2232: extracting number length features and number change features from the territorial mark number classification data, thereby obtaining number length feature data and number change feature data;

[0160] Step S2233: Perform cluster calculation on the number length feature data and the number change feature data according to the classification quantity data, so as to obtain a blacklist number classification data set to send to the redis cache server.

[0161] Specifically, for example, the following territorial marks and their corresponding number of numbers: territorial mark A: 500 numbers, territorial mark B: 700 numbers, territorial mark C: 300 numbers. In step S2231, the system generates classified quantity data based on these data: for territorial mark A, the classified quantity data is 500, for territorial mark B, the classified quantity data is 700, and for territorial mark C, the classified quantity data is 300. In step S2232, the system extracts features from the territorial mark number classification data, including number length features and number change features. For example: for the number set of territorial mark A, the system extracts features with an average length of 10 digits and a coefficient of variation of 0.5. For the number set of territorial mark B, the system extracts features with an average length of 11 digits and a coefficient of variation of 0.3. For the number set of territorial mark C, the system extracts features with an average length of 9 digits and a coefficient of variation of 0.6. In step S2233, the system performs clustering calculations based on the classification quantity data and feature data to obtain the final blacklist number classification data set. For example, the system can cluster numbers with similar features and classification quantities to obtain a set of similar numbers. Finally, the system sends this clustered blacklist number classification data set to the redis cache server for subsequent use and query.

[0162] In the present invention, through step S2232, comprehensive features including number length features and number change features are extracted from the location-marked number classification data, enriching the description and classification basis of the number. Step S2233 uses the classification quantity data to perform cluster calculation on the feature data, clustering similar numbers together, and improving the accuracy and comprehensiveness of the classification. The final blacklist number classification data set is sent to the redis cache server, which realizes fast storage and query and improves the response speed of the system. By introducing different features and combining the clustering method, the classification is made more accurate and refined, and the misjudgment rate is reduced.

[0163] Preferably, the classification quantity data generated in step S2231 is calculated by a number characteristic classification quantity calculation formula, wherein the number characteristic classification quantity calculation formula is specifically:

[0164]

[0165] Q is the number of classification data, θ is the weight coefficient of the number of classification data of the territorial mark number, b is the constant term of the territorial mark base, k is the number of classification data of the territorial mark number, δ is the weight coefficient of the number of types of classification data of the territorial mark number, m is the number of types of classification data of the territorial mark number, x is the classification data of the territorial mark number, f(x) is the mapping item of the territorial mark number characteristics, p i Probability distribution data for territorial mark number classification data.

[0166] The present invention constructs a number characteristic classification quantity calculation formula. The classification quantity data provided by the formula is further clustered to optimize the classification results of blacklist numbers and improve the filtering accuracy. The present invention makes dynamic adjustments according to actual conditions and can determine appropriate weight coefficients according to the distribution of experimental or actual data, thereby improving the flexibility and adaptability of calculation. The introduced characteristic mapping term f(x) makes the calculation process more comprehensive and more sensitive to characteristic changes. Where p i It is the probability distribution data of the territorial mark number classification data, which represents the probability distribution of each classification. It is usually represented by the frequency of data occurrence / total number of data, and can also be replaced by a probability distribution function. The weight coefficients (θ, δ) of different parameters can be used to balance the influence of quantity data and type quantity data, making the evaluation more reasonable and comprehensive. Conventional technical means are limited by their dependence on specific characteristics or insufficient trade-offs, while the present invention can more comprehensively reflect the characteristics of the number of classifications through dynamic adjustment of parameters, thereby improving the classification accuracy and enhancing the effect of blacklist filtering.

[0167] Preferably, step 51 specifically includes:

[0168] Step 511: when it is determined that the classification mark matching result data is the matching success data, the number data to be detected and the blacklist number data corresponding to the blacklist number classification data set corresponding to the classification mark matching result data are matched to obtain the number matching result data;

[0169] Specifically, for example, a regular expression is used to perform number matching.

[0170] Specifically, for example, a one-to-one correspondence principle is used to perform a complete match.

[0171] Step 512: When it is determined that the number matching result data is number matching success result data, the number matching result data is determined as blacklist matching result data;

[0172] Specifically, for example, if the number is successfully matched in step 511, the matching result can be determined as blacklist matching result data.

[0173] Step 513: when it is determined that the number matching result data is number matching failure result data, historical data acquisition is performed according to the number data to be detected, thereby acquiring historical call data;

[0174] Specifically, for example, historical call data is obtained from a database.

[0175] Step 514: Match the preset call behavior parameters according to the call record data and the interaction record data of the historical call data, so as to obtain blacklist matching result data.

[0176] Specifically, for example, matching is performed based on call duration and call frequency. def match_call_behavior(history_data):for call_record in history_data:call_duration=call_record['duration']call_frequency=call_record['frequency']if call_duration>threshold_duration and call_frequency>threshold_frequency:return "Call behavior matching is successful", return "No matching result".

[0177] The present invention introduces a multi-dimensional matching strategy, which not only takes into account the direct matching of numbers, but also takes into account the historical call records, thereby improving the accuracy of matching. Step S513 makes full use of the historical call data of the number to be detected, provides more information for matching, expands the role of blacklist screening in the actual process, and increases the comprehensiveness of matching. Step S514 introduces preset call behavior parameters, which can be flexibly adjusted according to actual needs, making the matching more targeted. By comprehensively considering information in multiple dimensions, the accuracy of blacklist matching is improved and the misjudgment rate is reduced.

[0178] The beneficial effects of the present invention are: blacklist number data is quickly obtained through a preset database, and a Redis cache server is used for efficient classification and marking. This ensures the real-time and efficiency of the whole process. The method allows the classification and marking rules to be flexibly preset and adjusted according to specific needs to adapt to blacklist filtering in different scenarios and needs. The number to be detected is matched with the blacklist classification data set, and the matching result is classified and marked, so as to accurately distinguish the data of successful matching and failed matching. When the classification mark matching result data is a match failure, the number to be detected is determined as whitelist detection number data, so as to avoid misjudgment of legitimate numbers. For the data that successfully matches, multi-dimensional matching is performed, including historical record matching and number matching, so as to improve the filtering accuracy and reliability. For the data that failed to match, by obtaining historical call data, matching is performed in combination with preset call behavior parameters to determine whether it is blacklist detection number data. It is suitable for processing large-scale blacklist number data, and the scalability and applicability of the method are guaranteed. Through multi-dimensional matching and historical data analysis, the possibility of misjudgment is reduced, the accuracy of filtering is improved, and the communication security of users is protected.

[0179] Therefore, from any point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the attached application documents rather than the above description, and it is intended that all changes falling within the meaning and scope of equivalent elements of the application documents are included in the present invention.

[0180] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A big data blacklist filtering method, characterized in that: The following steps are involved: Step S1: Obtain blacklist number data from a preset database to obtain blacklist number data; Step S2: Generate blacklist sub-number data according to the blacklist number data, and classify and mark the blacklist number data according to the blacklist sub-number data, so as to obtain a blacklist number classification data set to send to the redis cache server; Step S3: obtaining the data of the number to be detected, and extracting the classification mark of the number to be detected, thereby obtaining the classification mark data of the number to be detected, so as to match the classification mark data of the number to be detected with the classification data set of the blacklist number, and obtain the classification mark matching result data, wherein the classification mark matching result data includes matching success data and matching failure data; Step S4: when it is determined that the classification mark matching result data is matching failure data, the number data to be detected is determined as whitelist detection number data; Step S5: When it is determined that the classification mark matching result data is successfully matched data, the blacklist number data corresponding to the blacklist number classification data set corresponding to the classification mark matching result data and the number data to be detected are matched to obtain the blacklist matching result data; If it is determined that the blacklist matching result data does not exist, the number data to be detected is determined as whitelist detection number data; If it is determined that the blacklist matching result data is matching result data, the to-be-detected number data is determined as blacklist detection number data to perform a number filtering operation; Step S2 is specifically as follows: Step S21: extracting the number digit data of the blacklist number data by using the preset sub-number threshold data, thereby obtaining the blacklist sub-number data; Step S22: classify and mark the blacklist number data according to the blacklist sub-number data, thereby obtaining a blacklist number classification data set to send to the redis cache server; Step S21 is specifically as follows: Step S211: extracting the number digit data of the blacklist number data by using the preset first sub-number threshold data, thereby obtaining the first preliminary blacklist sub-number data; Step S212: extracting the number digit data of the blacklist number data by using the preset second sub-number threshold data, thereby obtaining the second preliminary blacklist sub-number data; Step S213: performing number information entropy calculation on the first preliminary blacklist sub-number data and the second preliminary blacklist sub-number data, thereby obtaining first sub-number information entropy data and second sub-number information entropy data respectively; wherein the number information entropy calculation is performed by using a number information entropy calculation formula, and the number information entropy calculation formula is specifically: ; is the sub-number information entropy data, is the number sequence item, The number of different numbers in the blacklist sub-number data, For the The probability data of the number appearing, The blacklist number data Sub-number data, For the The weight coefficient of the probability data of the number appearing, For the The weight coefficient of the location data of the number, For the The location data of the number, For the The weight coefficient of the length data of the number, For the The length of the number, For the The weight coefficient of the characteristic data of the number, For the Characteristic data of the number, is the error adjustment item of the sub-number information entropy data; Step S214: when it is determined that the first sub-number information entropy data is greater than or equal to the second sub-number information entropy data, the first sub-number information entropy data is determined as blacklist sub-number data; Step S215: When it is determined that the first sub-number information entropy data is less than the second sub-number information entropy data, the second sub-number information entropy data is determined as blacklist sub-number data.

2. The method according to claim 1, characterized in that: Step S1 is specifically as follows: Step S11: Obtain database request connection data; Step S12: Connecting to the database according to the database request connection data, thereby obtaining database connection data; Step S13: acquiring data according to the blacklist number collection requirement data in the database request connection data and the database connection data, thereby acquiring the blacklist number data.

3. The method according to claim 1, characterized in that Step S3 is specifically as follows: Step S31: obtaining the data of the number to be detected, and extracting the number position data of the number to be detected through the preset sub-number threshold data, thereby obtaining the classification mark data of the number to be detected; Step S32: matching is performed according to the classification mark data of the number to be detected and the classification data set of the blacklist number to obtain classification mark matching result data, wherein the classification mark matching result data includes matching success data and matching failure data.

4. The method according to claim 1, characterized in that Step S5 is specifically as follows: Step S51: when it is determined that the classification mark matching result data is the matching success data, multi-dimensional matching is performed on the blacklist number data corresponding to the blacklist number classification data set corresponding to the classification mark matching result data, so as to obtain the blacklist matching result data, wherein the multi-dimensional matching includes historical record matching and number matching; Step S52: if it is determined that the blacklist matching result data does not exist, the number data to be detected is determined as whitelist detection number data; Step S53: If it is determined that the blacklist matching result data is matching result data, the to-be-detected number data is determined as blacklist detection number data to perform a number filtering operation.

5. The method according to claim 2, characterized in that: Step S12 is specifically as follows: Step S121: obtaining system current status data, wherein the system current status data includes system load data, database server performance data, and connection request frequency data; Step S122: Select a connection pool strategy based on the current system status data, thereby obtaining connection pool strategy data; Step S123: Create a connection pool according to the connection pool policy data, thereby obtaining real-time connection pool creation data; Step S124: Create data in the real-time connection pool to connect to the database, thereby obtaining database connection data.

6. The method according to claim 1, characterized in that Step S22 is specifically as follows: Step S221: classify and mark the blacklist number data according to the blacklist sub-number data, thereby obtaining a preliminary blacklist number classification data set; Step S222: performing location marking on the preliminary blacklist number classification data in the preliminary blacklist number classification data set, thereby obtaining a location marked number classification data set; Step S223: Perform maximum error clustering calculation on the location mark number classification data in the location mark number classification data set, so as to obtain a blacklist number classification data set to send to the redis cache server.

7. The method according to claim 6, characterized in that Step S223 is specifically as follows: Step S2231: generating classification quantity data according to the territorial mark number classification data in the territorial mark number classification data set; Step S2232: extracting number length features and number change features from the territorial mark number classification data, thereby obtaining number length feature data and number change feature data; Step S2233: Perform cluster calculation on the number length feature data and the number change feature data according to the classification quantity data, so as to obtain a blacklist number classification data set to send to the redis cache server.

8. The method according to claim 7, characterized in that The classification quantity data generated in step S2231 is calculated by the number characteristic classification quantity calculation formula, wherein the number characteristic classification quantity calculation formula is specifically: ； For categorical quantity data, The weight coefficient of the quantity data of the territorial mark number classification data, is the base constant term of the territorial mark, Quantity data for classification data of territory marking numbers, The weight coefficient of the type and quantity data of the territorial mark number classification data, The number of categories of territorial mark number classification data, Classify data for territorial marking numbers, It is a mapping item of the local marker number characteristic. Probability distribution data for territorial mark number classification data.

9. The method according to claim 3, characterized in that: Step 51 is specifically as follows: Step 511: when it is determined that the classification mark matching result data is successfully matched data, the number data to be detected and the blacklist number data corresponding to the blacklist number classification data set corresponding to the classification mark matching result data are matched to obtain the number matching result data; Step 512: When it is determined that the number matching result data is number matching success result data, the number matching result data is determined as blacklist matching result data; Step 513: when it is determined that the number matching result data is number matching failure result data, historical data acquisition is performed according to the number data to be detected, thereby acquiring historical call data; Step 514: Match the preset call behavior parameters according to the call record data and the interaction record data of the historical call data, so as to obtain blacklist matching result data.

Images