Data filtering method and device, equipment, storage medium and program product

Abstract

The invention relates to a data filtering method and device, equipment, a storage medium and a program product. The method comprises the steps of obtaining to-be-filtered data, performing Hash processing on the to-be-filtered data to obtain a target Hash value, then determining a target memory module corresponding to the target Hash value according to the target Hash value, and performing data filtering processing on the target Hash value according to a target filtering rule fragment corresponding to the target memory module, the memory modules are in one-to-one correspondence with different filtering rule fragments. By adopting the method, resource consumption can be reduced.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data filtering method, device, equipment, storage medium and program product. Background Art

[0002] With the development of Internet technology, information security issues have become increasingly prominent. New types of attacks such as network worms and computer viruses are emerging in an endless stream. Therefore, it is necessary to filter the received data to screen out abnormal data that poses a threat to information security.

[0003] Traditional data filtering technologies, such as Bloom Filter (BF), store complete filtering rules in a filtering rule set in multiple memories through hash compression. The compressed filtering rules support hash queries of data, so that data can be matched with the complete filtering rules in each memory respectively to achieve data filtering.

[0004] However, the above method has the problem of high resource consumption. Summary of the invention

[0005] Based on this, it is necessary to provide a data filtering method, device, equipment, storage medium and program product that can reduce resource consumption in response to the above technical problems.

[0006] In a first aspect, the present application provides a data filtering method, comprising:

[0007] Obtain the data to be filtered, and perform hash processing on the data to be filtered to obtain the target hash value;

[0008] Determine a target memory module corresponding to the target hash value according to the target hash value, and perform data filtering processing on the target hash value according to a target filtering rule fragment corresponding to the target memory module to obtain a data filtering result;

[0009] Among them, each memory module corresponds to a different filtering rule fragment.

[0010] In the above embodiment, by decomposing the target filtering rule into different filtering rule fragments, the different filtering rule fragments are respectively stored in multiple memory modules, and each memory module is combined to store only one complete filtering rule. In the process of data filtering, the corresponding target memory module is found according to the target hash value of the data to be filtered, and the target filtering rule fragment corresponding to the target memory module can be used to implement data filtering of the data to be filtered. This avoids the problem of high resource consumption in traditional technology of storing complete filtering rules in multiple memories and then matching filtering rules for the data to be filtered respectively through the complete filtering rules stored in multiple memories. The embodiment of the present application reduces resource consumption in the data filtering process.

[0011] In one embodiment, the target hash value includes a query address, and data filtering processing is performed on the target hash value according to a target filtering rule fragment corresponding to the target memory module to obtain a data filtering result, including:

[0012] According to the query address, find and obtain the target bit value of the address bit corresponding to the query address in the target memory module;

[0013] According to the target bit value, the data filtering result is determined.

[0014] In the above embodiment, the server finds the target bit value of the address bit corresponding to the query address according to the query address included in the target hash value, and determines whether the data to be filtered belongs to the target filtering rule fragment through the target bit value, thereby improving the efficiency of data filtering.

[0015] In one embodiment, determining a data filtering result according to a target bit value includes:

[0016] If the target bit value is the first value, determining the data filtering result is that the target hash value successfully matches the target filtering rule fragment;

[0017] If the target bit value is the second value, it is determined that the data filtering result is that the target hash value fails to match the target filtering rule fragment, and the first value and the second value are different.

[0018] In the above embodiment, the first value and the second value are used to determine whether the target Hash value and the target filtering rule fragment are matched successfully. The determination condition is simple, so that the efficiency of data filtering is improved.

[0019] In one embodiment, the target hash value includes an index address, and determining a target memory module corresponding to the target hash value according to the target hash value includes:

[0020] According to the index address, determining a memory module identifier corresponding to the index address;

[0021] According to the memory module identifier, a target memory module corresponding to the memory module identifier is determined.

[0022] In the above embodiment, the server determines the target filtering rule fragment to be filtered by the index address included in the target Hash value, thereby improving the efficiency of data filtering.

[0023] In one embodiment, the method further comprises:

[0024] Acquire a filtering rule set, and perform hash processing on the filtering rules included in the filtering rule set to obtain multiple rule hash values;

[0025] Adjust each bit value in the initial filtering rule table according to the hash value of each rule to obtain the target filtering rule table;

[0026] According to the target filtering rule table, the bit values ​​of the corresponding address bits in each memory module are configured.

[0027] In the above embodiment, the server determines the target filter rule table based on the filter rule set, and then configures the bit values ​​of the corresponding address bits in each memory module according to the target filter rule table, thereby dividing the target filter rule table into multiple filter rule fragments and mapping them to multiple memory modules, thereby reducing the consumption of memory resources.

[0028] In one of the embodiments, data filtering is performed on the target hash value according to the target filtering rule fragment corresponding to the target memory module to obtain a data filtering result, including:

[0029] Performing initial data filtering processing on the target hash value according to the target data filtering rule fragment corresponding to the target memory module to obtain an initial data filtering result;

[0030] If the initial data filtering result does not meet the preset conditions, the initial data filtering result is continuously subjected to lower-level data filtering processing until a data filtering result meeting the preset conditions is obtained, and the preset conditions are related to the preset data filtering ratio.

[0031] In the above embodiment, by setting a multi-level data filtering process, the data filtering result meets the preset conditions, thereby improving the accuracy of data filtering.

[0032] In a second aspect, the present application also provides a data filtering device, comprising:

[0033] The acquisition module is used to acquire the data to be filtered and perform hash processing on the data to be filtered to obtain the target hash value;

[0034] A filtering module, used to determine a target memory module corresponding to the target hash value according to the target hash value, and perform data filtering processing on the target hash value according to a target filtering rule fragment corresponding to the target memory module to obtain a data filtering result;

[0035] Among them, each memory module corresponds to a different filtering rule fragment.

[0036] In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of the first aspect when executing the computer program.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method of the first aspect described above.

[0038] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method in the first aspect are implemented.

[0039] The above-mentioned data filtering method, device, equipment, storage medium and program product obtain the data to be filtered and perform hash processing on the data to be filtered to obtain the target hash value. Then, the target memory module corresponding to the target hash value is determined according to the target hash value, and the target hash value is subjected to data filtering processing according to the target filtering rule fragment corresponding to the target memory module to obtain the data filtering result, wherein each memory module corresponds to different filtering rule fragments one by one. In this way, by decomposing the target filtering rule into different filtering rule fragments, the different filtering rule fragments are respectively stored in multiple memory modules, and each memory module only needs to store the complete filtering rule once when combined. In the process of data filtering, the corresponding target memory module is found according to the target hash value of the data to be filtered, and the target filtering rule fragment corresponding to the target memory module can be used to implement data filtering of the data to be filtered, thereby avoiding the problem of high resource consumption in the traditional technology of storing complete filtering rules in multiple memories and then matching filtering rules with the data to be filtered respectively according to the complete filtering rules stored in multiple memories. The embodiment of the present application reduces the resource consumption in the data filtering process. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 Specific architecture for traditional hardware DPI technology implementation scheme;

[0042] Figure 2 It is a traditional single-channel BF filter structure;

[0043] Figure 3 An application environment diagram of a data filtering method in an embodiment;

[0044] Figure 4 A schematic diagram of a data filtering method in one embodiment;

[0045] Figure 5 is a flow chart of step 402 in another embodiment;

[0046] Figure 6 is a flow chart of step 402 in another embodiment;

[0047] Figure 7 It is a flowchart of a process in which a server configures the bit value of a corresponding address bit in each memory module according to a filtering rule in another embodiment;

[0048] Figure 8 is a flow chart of step 402 in another embodiment;

[0049] Fig. 9 is a structural block diagram of a data filtering device in one embodiment;

[0050] Fig.10 is a structural block diagram of a data filtering device in one embodiment;

[0051] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] With the development of Internet technology, information security issues have become increasingly prominent. New attacks such as network worms, computer viruses, and botnets are emerging one after another. In order to prevent these virus data from invading and hijacking computer systems and user sensitive privacy information, it is necessary to check the received data. Traditional Internet security defense technologies, such as product firewalls, usually only check the header of the received data packet, but not all the received data, so some virus data cannot be checked out, resulting in incomplete defense. Therefore, Deep Packet Inspection (DPI) technology was created.

[0054] DPI uses a feature matching algorithm to match the data received by the server with the preset filtering rules, thereby checking out the data that meets the preset filtering rules, and these data that meet the preset filtering rules are virus data. Through DPI technology, data that meets the preset filtering rules can be checked out, thereby filtering out virus data.

[0055] DPI technology is an intensive computing technology, which is mainly used in the key data paths of high-speed routers. The inspected data is massive and the data scale of the filtering rules is also very large. Therefore, DPI technology has certain requirements for memory resources.

[0056] In order to meet the performance requirements of high-speed data screening and storage of a large number of filtering rules, mainstream DPI technology implementation plans are divided into software DPI technology implementation plans based on software applications and hardware DPI technology implementation plans based on hardware applications.

[0057] Among them, software DPI technology implementation plans usually design serial feature matching algorithms for single-core platforms to develop deep packet inspection systems. Feature matching algorithms include the Nuth-Morris-Parrot algorithm, the Bayemore algorithm, and the Aho-Kroske algorithm and the Comantz-Walter algorithm in multi-pattern matching algorithms. However, with the rapid growth of network bandwidth and business traffic and the increasing number of filtering rules, it is required to analyze, detect and reassemble network traffic at extremely high speeds to avoid delays in the network. The software DPI technology implementation plans implemented by such feature matching algorithms can no longer meet the high performance requirements of data filtering.

[0058] The hardware DPI technology implementation plan first uses parallel BF filters to perform preliminary filtering on the large amount of data received, filters out data that is definitely not virus data, and detects suspicious data that may be virus data. At this time, the data size of the suspicious data is greatly reduced, and the suspicious data is accurately filtered through TCAM (ternary content addressable memory) to detect virus data.

[0059] TCAM is a ternary content addressable memory that can perform precise matching searches on data. Therefore, it is often used in hardware DPI technology implementation schemes to accurately process suspicious data obtained after preliminary filtering by the BF filter.

[0060] The BF filter is a space-efficient data structure that stores the complete filtering rules in the filtering rule set in multiple memories through hash compression. The compressed filtering rules support hash queries of data, so that the data can be matched with the complete filtering rules in each memory respectively. However, since the BF filter has certain false positive misjudgments when filtering data, that is, the BF filter can only filter out data that is definitely not virus data. Therefore, the BF filter is used in the hardware DPI technology implementation plan to perform preliminary filtering processing on the received data.

[0061] Refer to Figure 1 The specific architecture of the hardware DPI technology implementation scheme shown is, for example, when the processing clock of the FPGA chip is 400Mbps and 1Byte is processed at a time, the processing capacity of a single channel is about 400MB, or 3.2Gb. If you want to filter 400Gb of data to be filtered per second, you need to transmit 400Gb of data to be filtered through 128 channels, and the data to be filtered is sent to the BF filter corresponding to each channel through 128 channels.

[0062] Since the query capability of the subsequent TCAM is limited, the data size of the suspicious data obtained after the initial filtering of the data to be filtered needs to meet the query capability of the TCAM. For example, the query capability of the TCAM is about 200Mpps, that is, 200 million bytes are queried per second, and the amount of data to be queried in the above example is 400Gb, which requires that the data filtering ratio of each BF filter be less than 4‰.

[0063] For the conventional single-channel BF filter in the above example, its structure is as follows: Figure 2 As shown, a feature rule set is stored in a memory (which can be a random access memory), the length of a memory is 65536 bits, the length of the filtering rule is 10000 bits, and the data filtering ratio of a memory is 15%. Three identical memories are set in parallel, and each memory stores complete filtering rules, so that 3.2Gb of data to be queried transmitted by a channel is initially filtered three times, so that the data filtering ratio is less than 4‰.

[0064] Using the above method, a BF filter of one channel requires 3 memories to store three complete filtering rules. If you want to process 400Gb of filtered data in parallel, 128 channel BF filters require a total of 384 memories, and the storage resource consumption is 24Mbit in total. This places extremely high performance requirements on FPGA and there is a problem of high resource consumption. With the advancement of network technology, there will be more and more filtered data and filtering rules to be filtered. The above hardware DPI technology implementation plan is difficult to achieve network security detection on existing FPGA chips.

[0065] In view of this, the embodiments of the present application provide a data filtering method, apparatus, device, storage medium and program product, which obtains the data to be filtered and performs hash processing on the data to be filtered to obtain a target hash value, then determines the target memory module corresponding to the target hash value according to the target hash value, and performs data filtering processing on the target hash value according to the target filtering rule fragment corresponding to the target memory module to obtain a data filtering result, wherein each memory module corresponds to a different filtering rule fragment one by one, so that by decomposing the target filtering rule into different filtering rule fragments, the different filtering rule fragments are respectively stored in multiple memory modules, and each memory module only needs to store the complete filtering rule once when combined. In the process of data filtering, the corresponding target memory module is found according to the target hash value of the data to be filtered, and the target filtering rule fragment corresponding to the target memory module can be used to implement data filtering of the data to be filtered, thereby avoiding the problem of high resource consumption in the traditional technology of storing complete filtering rules in multiple memories and then matching filtering rules with the data to be filtered respectively according to the complete filtering rules stored in the multiple memories. The embodiments of the present application reduce the resource consumption in the data filtering process.

[0066] The data filtering method provided in the embodiment of the present application can be applied to Figure 3 In the application environment shown. The data storage system can store the data that the server 302 needs to process. The data storage system can be integrated on the server 302, or it can be placed on the cloud or other network servers. Among them, the server 302 can be implemented as an independent server or a server cluster composed of multiple servers.

[0067] In an exemplary embodiment, Figure 4 As shown, a data filtering method is provided, which is applied to Figure 1 The server 302 in the example is used as an example to illustrate, and the following steps are included:

[0068] Step 401, obtain the data to be filtered, and perform hash processing on the data to be filtered to obtain a target hash value.

[0069] When the server is running, it receives a large amount of network data through multiple transmission channels. There may be abnormal data with information security hazards in these network data. Therefore, after the server receives the network data, the server needs to filter the network data.

[0070] Optionally, the server receives network data in real time, and according to a preset data length, whenever receiving network data of the preset data length, the server uses the network data of the preset data length as the data to be filtered; optionally, the server periodically receives network data, and divides the network data into multiple data to be filtered according to the preset data length. The manner in which the server obtains the data to be filtered is not specifically limited.

[0071] The preset data length can be 4 bytes or 3 bytes. Since the preset data length has a certain influence on the filtering ratio of data filtering, if the preset byte length is too short, such as 1 byte, it is easy to produce false hits during the data filtering process, and the filtering ratio is too high; if the preset byte length is too long, such as 5 bytes, it is easy to miss suspicious data during the data filtering process, and the filtering ratio is too low. Therefore, the preset data length is determined according to the actual application scenario and is not specifically limited here.

[0072] After receiving the data to be filtered, the server performs hash processing on the data to be filtered. In one possible implementation, the server performs hash compression on the data to be filtered according to a preset hash function to obtain a target hash value. Optionally, the server performs hash processing on the data to be filtered according to preset conditions to obtain a target hash value.

[0073] In a possible implementation, the target hash value is binary data obtained after the server performs hash processing on the data to be filtered. For a specific explanation of the target hash value, see the following embodiment.

[0074] Step 402 , determining a target memory module corresponding to the target hash value according to the target hash value, and performing data filtering processing on the target hash value according to a target filtering rule fragment corresponding to the target memory module to obtain a data filtering result.

[0075] The server is configured with a memory, which may be a random access memory (RAM). One memory is composed of a plurality of memory modules, and the memory module may be an M20K with independent read / write addresses and data buses.

[0076] In the embodiment of the present application, the number of memory modules used in the data filtering process can be determined according to the data to be filtered received by the server and the hardware limitation of the data transmission channel. For example, the server receives 400Gb of data to be filtered per second, and the data transmission capacity of the data transmission channel is 400MBs, that is, 3.2Gbs. Then, the number of data transmission channels is the amount of data to be filtered divided by the data transmission capacity of a single data transmission channel, that is, 125. In a possible implementation, in order to facilitate subsequent processing, the number of data transmission channels is set to 2. n That is, the number of data transmission channels is the factorial number of 2 greater than 125, that is, 2 7 That is, 128, so the number of memory modules is 128.

[0077] In a possible implementation, each memory module corresponds to a different filtering rule fragment.

[0078] The filtering rules are feature rules used to filter the data to be filtered. The filtering rules are used to characterize the features of the data to be filtered when it is virus data. The content of the filtering rules includes at least the virus data type, feature string, search starting position, response operation after matching, etc. The server performs hash processing on multiple filtering rules according to the hash processing method for the filtered data mentioned above to obtain the rule hash value corresponding to each filtering rule. Optionally, the server stores each filtering rule in each memory module according to certain mapping conditions and rule hash values; optionally, each memory module corresponds to a different identifier, and the server determines the identifier of the memory module corresponding to the rule hash value according to the rule hash value, and then stores each filtering rule in the corresponding memory module according to the identifier.

[0079] Regarding the memory size of each memory module, in a possible implementation manner, the memory size of each memory module is determined according to the number of filtering rules and a preset filtering ratio.

[0080] Exemplarily, the number of filtering rules is 100,000, and the preset filtering ratio is determined to be 5%, then approximately 2M of storage space is required, which is 100,000 divided by 5%. Taking the number of memory modules as 128 as an example, the storage space required for a single memory module is 15625 bits. For the convenience of calculation, the storage space of the memory module is set to a number greater than 15625 times the factorial of 2, that is, 16384 bits, which is 2 to the 14th power. Then the storage space required for a single memory module is 16384 bits, and each bit of the address bit corresponds to a bit value.

[0081] Regarding the method in which the server stores filtering rules in a memory module, taking a filtering rule as an example, optionally, the server performs hash processing on the filtering rule through a preset hash processing method to obtain a corresponding rule hash value, and each address bit corresponds to a different address bit identifier. The server determines the identifier of the address bit corresponding to the rule hash value based on the rule hash value corresponding to the filtering rule, and then determines the address bit corresponding to the rule hash value based on the identifier. After the server determines the address bit corresponding to the rule hash value, in a possible implementation manner, before storing the filtering rule, the corresponding bit value of each address bit is a preset first value, for example, it may be 0. After the server determines the address bit corresponding to the rule hash value, the bit value corresponding to the address bit is updated to a preset second value, for example, it may be 1.

[0082] Since the server inserts the filtering rules into each memory module based on the filtering hash value after hash compression of the filtering rules, and the hash processing method of the filtering rules is the same as the hash processing method of the data to be filtered, the server can determine the corresponding target memory module according to the target hash value corresponding to the data to be filtered, thereby determining the target filtering rule fragment corresponding to the target hash value.

[0083] Regarding a method for a server to determine a corresponding target memory module according to a target hash value corresponding to data to be filtered, in a possible implementation, the target hash value includes an index address, and the server can query the identifier corresponding to the corresponding target memory module according to the query address, and then determine the target memory module according to the identifier corresponding to the target memory module; Regarding a method for a server to perform data filtering processing on a target hash value according to a target filtering rule fragment corresponding to the target memory module, in a possible implementation, the target hash value also includes a query address, and the server can determine the address bit corresponding to the data to be filtered according to the query address, and then determine whether the data to be filtered successfully matches the filtering rule corresponding to the bit value according to the bit value corresponding to the address bit. Exemplarily, if the bit value is a preset first value, it means that the data to be filtered fails to match the filtering rule corresponding to the bit value, and the server filters out the data to be filtered, or if the bit value is a preset second value, it means that the data to be filtered successfully matches the filtering rule corresponding to the bit value, and the data to be filtered may comply with the filtering rule. At this time, the server retains the data to be filtered as one of the suspicious data.

[0084] In an embodiment of the present application, the hash processing method for hashing the data to be filtered can be the same as the hash processing method for processing the filtering rules, so as to obtain a target hash value with the same form and representational meaning as the rule hash value obtained after hashing the filtering rules, and then, according to the target hash value, a target filtering rule fragment that can filter the data to be filtered is selected from multiple memory modules to complete the data filtering process.

[0085] The above-mentioned data filtering method obtains the data to be filtered and performs hash processing on the data to be filtered to obtain a target hash value. Then, the target memory module corresponding to the target hash value is determined according to the target hash value, and the target hash value is subjected to data filtering processing according to the target filtering rule fragment corresponding to the target memory module to obtain a data filtering result, wherein each memory module corresponds to a different filtering rule fragment one by one. In this way, by decomposing the target filtering rule into different filtering rule fragments, the different filtering rule fragments are respectively stored in multiple memory modules, and each memory module only needs to store the complete filtering rule once when combined. In the process of data filtering, the corresponding target memory module is found according to the target hash value of the data to be filtered, and the target filtering rule fragment corresponding to the target memory module can be used to implement data filtering of the data to be filtered, thereby avoiding the problem of high resource consumption in the traditional technology of storing complete filtering rules in multiple memories and then matching filtering rules with the data to be filtered respectively according to the complete filtering rules stored in the multiple memories. The embodiment of the present application reduces the resource consumption in the data filtering process.

[0086] In one embodiment, based on the above Figure 4 The embodiment shown, see Figure 5 In this embodiment, the target hash value includes a query address. This embodiment involves a process of performing data filtering processing on the target hash value according to the target filtering rule fragment corresponding to the target memory module to obtain a data filtering result. Figure 5 As shown, step 402 includes:

[0087] Step 501: According to the query address, a target bit value of the address bit corresponding to the query address in the target memory module is found.

[0088] In one possible implementation, the query address represents the physical address corresponding to each address bit contained in the memory module; in another possible implementation, each address bit contained in the memory module corresponds to an identifier, and the query address can be used to represent the identifier of each address bit. Exemplarily, the storage space required for a single memory module is 16384 bits, and the 16384 address bits are identified. The identifiers corresponding to the address bits are 0 to 16383, and each identifier can be represented by a 14-bit binary code, which is the query address corresponding to the address bit. The server can determine the identifier of the target address bit corresponding to the data to be filtered based on the query address. For example, if the query address is 10001001001001, the identifier of the target address bit is 8777. According to the identifier 8777, it can be determined that the target address bit is address bit 8777.

[0089] A corresponding bit value is stored in each address bit, and each bit value is used to indicate whether the data to be filtered belongs to the target filtering rule segment.

[0090] Step 502: Determine the data filtering result according to the target bit value.

[0091] In one possible implementation, before the server stores the filtering rules into the memory module, it is necessary to initialize each memory module, that is, the bit value stored in the address bit contained in each memory module is a preset second value, for example, the second value may be 0. When the server determines that the bit value stored in the address bit needs to be updated based on the rule hash value corresponding to the filtering rule, the position of the address bit is updated according to the preset first value, for example, the first value may be 1.

[0092] Therefore, in one possible implementation, if the target bit value is the first value, it means that the target bit value has been updated, and the address bit corresponds to a target filtering rule, then the data filtering result is determined to be a successful match between the target hash value and the target filtering rule fragment; in another possible implementation, if the target bit value is the second value, it means that the target bit value is still the initialized second value, that is, the address bit is not stored in the target filtering rule, then the data filtering result is determined to be a failure to match the target hash value and the target filtering rule fragment, and the first value and the second value are different.

[0093] Thus, in the above embodiment, the server finds the target bit value of the address bit corresponding to the query address according to the query address included in the target hash value, and determines whether the data to be filtered belongs to the target filtering rule fragment through the target bit value, thereby improving the efficiency of data filtering.

[0094] In one embodiment, based on the above Figure 4 The embodiment shown, see Figure 6In this embodiment, the target hash value includes an index address, and this embodiment relates to a process of determining a target memory module corresponding to the target hash value according to the target hash value. Figure 6 As shown, the server can execute Figure 6 Steps 601 and 602 shown implement a process of determining a target memory module corresponding to a target hash value according to the target hash value:

[0095] Step 601: Determine the memory module identifier corresponding to the index address according to the index address.

[0096] In a possible implementation, each memory module corresponds to a different memory module identifier, and the memory module identifier can be represented by an index address, that is, the index address is used to characterize the memory module identifier corresponding to each memory module. Exemplarily, taking 128 memory modules as an example, each memory module is numbered from 0 to 127, and 128 is 2 to the 7th power, so each memory module can be represented by a 7-bit binary code, which is the index address, such as, if the index address is 0000000, then the memory module identifier corresponding to the index address is 0, and if the index address is 1001001, then the memory module identifier corresponding to the index address is 74.

[0097] Step 602: Determine a target memory module corresponding to the memory module identifier according to the memory module identifier.

[0098] When the server determines the memory module identifier corresponding to the index address according to the index address, the target memory module can be determined through the memory module identifier, and the to-be-filtered data can be filtered according to the rough target filtering rule fragment stored in the target memory module.

[0099] Thus, in the above embodiment, the server determines the target filtering rule fragment to be filtered by the index address included in the target hash value, thereby improving the efficiency of data filtering.

[0100] In one embodiment, based on the above Figure 5 The embodiment shown, see Figure 7 This embodiment relates to a process in which a server configures the bit values ​​of corresponding address bits in each memory module according to a filtering rule. Figure 7 As shown, the process includes:

[0101] Step 701: obtain a filtering rule set, and perform hash processing on the filtering rules included in the filtering rule set to obtain multiple rule hash values.

[0102] The filtering rule set includes multiple filtering rules. In a possible implementation manner, the server performs hash processing on each filtering rule included in the filtering rule set to obtain multiple rule hash values.

[0103] This hash processing is the same as the hash processing method performed by the server on the data to be filtered mentioned above. For example, the server performs hash compression on each filtering rule included in the filtering rule set based on the first hash formula, and the subsequent hash processing method for the data to be filtered is also based on the first hash formula to hash compress the data to be filtered. In this way, the multiple rule hash values ​​obtained by the server after hashing each filtering rule are in the same form as the target hash value obtained by hashing the data to be filtered, and the meaning they represent is also the same, that is, a binary address.

[0104] Step 702: adjust each bit value in the initial filtering rule table according to the hash value of each rule to obtain a target filtering rule table.

[0105] In a possible implementation, the initial filtering rule table is a bit string table, and the bit value of each bit in the bit string table is a null value. The length of the initial filtering rule table is determined according to the number of filtering rules and a preset filtering ratio. For example, if the number of filtering rules is 100,000 and the preset filtering ratio is 5%, the length of the initial filtering rule table is 2M.

[0106] In a possible implementation, each bit in the initial filtering rule table corresponds to a binary address, and the number of bits of the binary address is determined according to the length of the initial filtering rule table. For example, the length of the initial filtering rule table is 2M, and 2M is in 2 20 ~2 21 Therefore, the number of bits of the binary address is 21 bits. Therefore, in order to enable each filtering rule to be mapped to the initial filtering rule table, the hash processing method mentioned above can compress the filtering rule into a 21-bit rule hash value. Similarly, the filtering rule can be compressed into a 21-bit target hash value.

[0107] In a possible implementation, the server can determine multiple target bits in the initial filtering rule table based on the rule hash value corresponding to each filtering rule, and then set the bit value corresponding to each target bit to a preset rule value, that is, set the bit value corresponding to each target bit to 1.

[0108] Step 703: configure the bit value of the corresponding address bit in each memory module according to the target filtering rule table.

[0109] In a possible implementation, the server divides the target filter rule table into multiple filter rule fragments according to the memory module identifier corresponding to each memory module and the binary address corresponding to each bit in the target filter rule table. Exemplarily, the number of memory modules is 64, and the memory module identifiers corresponding to each memory module are 0 to 63, and each memory module identifier corresponds to a 6-bit binary code. Therefore, the high 6 bits of the binary address corresponding to each bit in the target filter rule table are used as the segmentation condition to divide the target filter rule table into 64 filter rule fragments.

[0110] After the server divides the target filtering rule table into multiple filtering rule fragments, the bit value of the corresponding address bit in each memory module is configured according to the binary address corresponding to each bit on each filtering rule fragment and the binary address corresponding to each address bit in the memory module. Exemplarily, taking the length of the target filtering rule table as 2M and the number of memory modules as 64 as an example, it can be determined that the storage size required for each memory module is 31250 bits. For the convenience of calculation, the storage size of each memory module is 32768 bits, that is, 2 to the 15th power. Then, each address bit contained in the memory module corresponds to or a 15-bit binary address. At this time, the server uses the last 15 bits of the binary address corresponding to each bit on each filtering rule fragment as a mapping condition, and maps each bit on each filtering rule fragment to each address bit contained in the memory module.

[0111] In a possible implementation, the bit value of the address bit corresponding to the bit with a null value on the filtering rule fragment remains unchanged, and the bit value of the address bit corresponding to the bit with a preset rule value on the filtering rule fragment is updated to the first value.

[0112] In this way, through the above embodiment, the server determines the target filter rule table according to the filter rule set, and then configures the bit value of the corresponding address bit in each memory module according to the target filter rule table, thereby dividing the target filter rule table into multiple filter rule fragments and mapping them to multiple memory modules, thereby reducing the consumption of memory resources.

[0113] In one embodiment, based on the above Figure 4 The embodiment shown, see Figure 8 This embodiment relates to a process of performing data filtering processing on a target hash value according to a target filtering rule fragment corresponding to a target memory module to obtain a data filtering result. Figure 8 As shown, step 402 includes:

[0114] Step 801 , performing initial data filtering processing on a target hash value according to a target data filtering rule fragment corresponding to a target memory module to obtain an initial data filtering result.

[0115] In one possible implementation, the data filtering process includes multiple levels, and the first level of data filtering process is the initial data filtering process. After the server performs initial data filtering process on the multiple received data to be filtered, multiple initial data filtering results are obtained. The initial data filtering results are used to characterize whether the data to be filtered matches the filtering rules. If they match, it means that the data to be filtered is suspicious data, and the server retains the data to be filtered. If they do not match, it means that the data to be filtered does not belong to the filtering rule set, and the server filters out the data to be filtered. Finally, the number of data to be filtered retained is the product of the total number of data to be filtered and the filtering ratio. For example, the number of data to be filtered is 400Gb, and the filtering ratio of the first-level initial data filtering process is 5%. In theory, the number of data to be filtered retained is 20Gb.

[0116] Step 802: If the initial data filtering result does not meet the preset conditions, continue to perform lower-level data filtering processing on the initial data filtering result until a data filtering result that meets the preset conditions is obtained.

[0117] In a possible implementation, the server judges the initial data filtering result according to a preset condition, wherein the preset condition is related to a preset data filtering ratio. For example, if the preset condition is that the filtering ratio reaches 4‰, the server judges the initial data filtering result according to the preset condition. If the filtering ratio after the initial data filtering process is less than 4‰, it means that the initial data filtering result meets the preset condition and the filtering process is completed. The server uses the data to be filtered retained by the initial filtering process as the data filtering result; if the filtering ratio after the initial data filtering process is less than or greater than 4‰, for example, in the above embodiment, the filtering ratio of the initial data filtering process is 5%, and the amount of data to be filtered retained is 20Gb, then the server needs to continue to perform data filtering on the data to be filtered retained after the initial data filtering process. Since the filtering rule set has not changed, the filtering ratio of the next level is still 5%, and the filtering ratio of the data filtering result after two data filtering processes is 2.5‰, which is less than the preset condition of 4‰. At this time, the data to be filtered retained after the two data filtering processes is used as the data filtering result.

[0118] It can be understood that after the initial data filtering process, the amount of data to be filtered that is retained is greatly reduced. When the server performs secondary data filtering on the data to be filtered that is retained by the initial data filtering process, the required data transmission channel resources are also reduced. For example, the data to be filtered for the initial data filtering process is 400Gb in total, and the transmission capacity of a single data transmission channel is 3.2Gbs. The initial data filtering process requires a total of 128 data transmission channels. When the filtering ratio is 5%, the amount of data to be filtered received during the secondary data filtering process is 20Gb, and the number of data transmission channels required is 8.

[0119] Thus, in the above embodiment, by setting up a multi-level data filtering process, the data filtering result meets the preset conditions, thereby improving the accuracy of data filtering.

[0120] In one embodiment, an exemplary data filtering method is provided, the method comprising:

[0121] Step a: obtaining a filtering rule set, and performing hash processing on the filtering rules included in the filtering rule set to obtain multiple rule hash values.

[0122] Step b: adjusting each bit value in the initial filtering rule table according to the hash value of each rule to obtain a target filtering rule table.

[0123] Step c: configuring the bit value of the corresponding address bit in each memory module according to the target filtering rule table.

[0124] Step d: Obtain the data to be filtered, and perform hash processing on the data to be filtered to obtain a target hash value.

[0125] The target hash value includes the index address.

[0126] Step e: determining the memory module identifier corresponding to the index address according to the index address.

[0127] Step f: determining a target memory module corresponding to the memory module identifier according to the memory module identifier.

[0128] Among them, the target hash value includes the query address.

[0129] Step g: according to the query address, find out the target bit value of the address bit corresponding to the query address in the target memory module.

[0130] Step h: if the target bit value is the first value, then determining the data filtering result is that the target hash value successfully matches the target filtering rule fragment.

[0131] Step i: if the target bit value is the second value, then determine that the data filtering result is that the target hash value fails to match the target filtering rule fragment, and the first value and the second value are different.

[0132] Among them, each memory module corresponds to a different filtering rule fragment.

[0133] Step j: performing initial data filtering processing on the target hash value according to the target data filtering rule fragment corresponding to the target memory module to obtain an initial data filtering result.

[0134] Step k: if the initial data filtering result does not meet the preset conditions, continue to perform lower-level data filtering processing on the initial data filtering result until a data filtering result that meets the preset conditions is obtained, and the preset conditions are related to the preset data filtering ratio.

[0135] The following, combined Fig. 9 , the implementation process of the data filtering method of the embodiment of the present application is exemplarily described.

[0136] 1) The data to be filtered enters the hash compression module through the data input channel.

[0137] 2) The hash compression module performs hash processing on the data to be filtered to obtain the target hash value.

[0138] 3) According to the index address, determine the memory module identifier corresponding to the index address.

[0139] 4) The matrix switching module determines the target memory module corresponding to the memory module identifier according to the memory module identifier, and then transmits the data to be filtered to the corresponding target memory module through the corresponding data transmission channel.

[0140] 5) The target memory module searches for the target bit value of the address bit corresponding to the query address in the target memory module according to the query address. If the target bit value is the first value, the data filtering result is determined to be a successful match between the target hash value and the target filtering rule fragment; if the target bit value is the second value, the data filtering result is determined to be a failure to match between the target hash value and the target filtering rule fragment, and the first value and the second value are different.

[0141] 6) The target memory module outputs the data filtering results.

[0142] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0143] Based on the same inventive concept, the embodiment of the present application also provides a data filtering device for implementing the data filtering method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data filtering device embodiments provided below can refer to the limitations on the data filtering method above, and will not be repeated here.

[0144] In an exemplary embodiment, as shown in Figure 1, a data filtering device is provided, including: an acquisition module 1001 and a filtering module 1002, wherein:

[0145] The acquisition module 1001 is used to acquire the data to be filtered and perform hash processing on the data to be filtered to obtain a target hash value;

[0146] A filtering module 1002 is used to determine a target memory module corresponding to the target hash value according to the target hash value, and perform data filtering processing on the target hash value according to a target filtering rule fragment corresponding to the target memory module to obtain a data filtering result;

[0147] Among them, each memory module corresponds to a different filtering rule fragment.

[0148] In one embodiment, the filtering module 1002 includes:

[0149] A search unit, used to search for a target bit value of an address bit corresponding to the query address in a target memory module according to the query address;

[0150] The determination unit is used for the user to determine the data filtering result according to the target bit value.

[0151] In one embodiment, the determining unit is further configured to:

[0152] If the target bit value is the first value, determining the data filtering result is that the target hash value successfully matches the target filtering rule fragment;

[0153] If the target bit value is the second value, it is determined that the data filtering result is that the target hash value fails to match the target filtering rule fragment, and the first value and the second value are different.

[0154] In one embodiment, the filtering module 1002 further includes:

[0155] A memory module identification confirmation unit, used to determine the memory module identification corresponding to the index address according to the index address;

[0156] The target memory module confirmation unit is used to determine the target memory module corresponding to the memory module identifier according to the memory module identifier.

[0157] In one embodiment, the data filtering device further includes:

[0158] A rule hash unit, used to obtain a filter rule set and perform hash processing on the filter rules included in the filter rule set to obtain multiple rule hash values;

[0159] An adjustment unit, used to adjust each bit value in the initial filtering rule table according to each rule hash value to obtain a target filtering rule table;

[0160] The configuration unit is used to configure the bit value of the corresponding address bit in each memory module according to the target filtering rule table.

[0161] In one embodiment, the filtering module 1002 further includes:

[0162] An initial data filtering processing unit, used to perform initial data filtering processing on the target hash value according to the target data filtering rule fragment corresponding to the target memory module, to obtain an initial data filtering result;

[0163] The lower level filtering unit is used to continue to perform lower level data filtering processing on the initial data filtering result if the initial data filtering result does not meet the preset conditions, until a data filtering result that meets the preset conditions is obtained, and the preset conditions are related to the preset data filtering ratio.

[0164] Each module in the above data filtering device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0165] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.11As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data filtering data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data filtering method is implemented.

[0166] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0167] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0168] Obtain the data to be filtered, and perform hash processing on the data to be filtered to obtain the target hash value;

[0169] Determine a target memory module corresponding to the target hash value according to the target hash value, and perform data filtering processing on the target hash value according to a target filtering rule fragment corresponding to the target memory module to obtain a data filtering result;

[0170] Among them, each memory module corresponds to a different filtering rule fragment.

[0171] In one embodiment, the target hash value includes a query address, and the processor further implements the following steps when executing the computer program:

[0172] According to the query address, find and obtain the target bit value of the address bit corresponding to the query address in the target memory module;

[0173] According to the target bit value, the data filtering result is determined.

[0174] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0175] If the target bit value is the first value, determining the data filtering result is that the target hash value successfully matches the target filtering rule fragment;

[0176] If the target bit value is the second value, it is determined that the data filtering result is that the target hash value fails to match the target filtering rule fragment, and the first value and the second value are different.

[0177] In one embodiment, the target hash value includes an index address, and the processor further implements the following steps when executing the computer program:

[0178] According to the index address, determining a memory module identifier corresponding to the index address;

[0179] According to the memory module identifier, a target memory module corresponding to the memory module identifier is determined.

[0180] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0181] Acquire a filtering rule set, and perform hash processing on the filtering rules included in the filtering rule set to obtain multiple rule hash values;

[0182] Adjust each bit value in the initial filtering rule table according to the hash value of each rule to obtain the target filtering rule table;

[0183] According to the target filtering rule table, the bit values ​​of the corresponding address bits in each memory module are configured.

[0184] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0185] Performing initial data filtering processing on the target hash value according to the target data filtering rule fragment corresponding to the target memory module to obtain an initial data filtering result;

[0186] If the initial data filtering result does not meet the preset conditions, the initial data filtering result is continuously subjected to lower-level data filtering processing until a data filtering result meeting the preset conditions is obtained, and the preset conditions are related to the preset data filtering ratio.

[0187] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0188] Obtain the data to be filtered, and perform hash processing on the data to be filtered to obtain the target hash value;

[0189] Determine a target memory module corresponding to the target hash value according to the target hash value, and perform data filtering processing on the target hash value according to a target filtering rule fragment corresponding to the target memory module to obtain a data filtering result;

[0190] Among them, each memory module corresponds to a different filtering rule fragment.

[0191] In one embodiment, the target hash value includes a query address, and the computer program when executed by a processor further implements the following steps:

[0192] According to the query address, find and obtain the target bit value of the address bit corresponding to the query address in the target memory module;

[0193] According to the target bit value, the data filtering result is determined.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0195] If the target bit value is the first value, determining the data filtering result is that the target hash value successfully matches the target filtering rule fragment;

[0196] If the target bit value is the second value, it is determined that the data filtering result is that the target hash value fails to match the target filtering rule fragment, and the first value and the second value are different.

[0197] In one embodiment, the target hash value includes an index address, and the computer program, when executed by a processor, further implements the following steps:

[0198] According to the index address, determining a memory module identifier corresponding to the index address;

[0199] According to the memory module identifier, a target memory module corresponding to the memory module identifier is determined.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0201] Acquire a filtering rule set, and perform hash processing on the filtering rules included in the filtering rule set to obtain multiple rule hash values;

[0202] Adjust each bit value in the initial filtering rule table according to the hash value of each rule to obtain the target filtering rule table;

[0203] According to the target filtering rule table, the bit values ​​of the corresponding address bits in each memory module are configured.

[0204] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0205] Performing initial data filtering processing on the target hash value according to the target data filtering rule fragment corresponding to the target memory module to obtain an initial data filtering result;

[0206] If the initial data filtering result does not meet the preset conditions, the initial data filtering result is continuously subjected to lower-level data filtering processing until a data filtering result meeting the preset conditions is obtained, and the preset conditions are related to the preset data filtering ratio.

[0207] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0208] Obtain the data to be filtered, and perform hash processing on the data to be filtered to obtain the target hash value;

[0209] Determine a target memory module corresponding to the target hash value according to the target hash value, and perform data filtering processing on the target hash value according to a target filtering rule fragment corresponding to the target memory module to obtain a data filtering result;

[0210] Among them, each memory module corresponds to a different filtering rule fragment.

[0211] In one embodiment, the target hash value includes a query address, and the computer program when executed by a processor further implements the following steps:

[0212] According to the query address, find and obtain the target bit value of the address bit corresponding to the query address in the target memory module;

[0213] According to the target bit value, the data filtering result is determined.

[0214] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0215] If the target bit value is the first value, determining the data filtering result is that the target hash value successfully matches the target filtering rule fragment;

[0216] If the target bit value is the second value, it is determined that the data filtering result is that the target hash value fails to match the target filtering rule fragment, and the first value and the second value are different.

[0217] In one embodiment, the target hash value includes an index address, and the computer program, when executed by a processor, further implements the following steps:

[0218] According to the index address, determining a memory module identifier corresponding to the index address;

[0219] According to the memory module identifier, a target memory module corresponding to the memory module identifier is determined.

[0220] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0221] Acquire a filtering rule set, and perform hash processing on the filtering rules included in the filtering rule set to obtain multiple rule hash values;

[0222] Adjust each bit value in the initial filtering rule table according to the hash value of each rule to obtain the target filtering rule table;

[0223] According to the target filtering rule table, the bit values ​​of the corresponding address bits in each memory module are configured.

[0224] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0225] Performing initial data filtering processing on the target hash value according to the target data filtering rule fragment corresponding to the target memory module to obtain an initial data filtering result;

[0226] If the initial data filtering result does not meet the preset conditions, the initial data filtering result is continuously subjected to lower-level data filtering processing until a data filtering result meeting the preset conditions is obtained, and the preset conditions are related to the preset data filtering ratio.

[0227] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0228] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0229] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0230] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data filtering method, characterized in that: The method comprises: Obtaining data to be filtered, and performing hash processing on the data to be filtered to obtain a target hash value; Determine a target memory module corresponding to the target hash value according to the target hash value, and perform data filtering processing on the target hash value according to a target filtering rule fragment corresponding to the target memory module to obtain a data filtering result; Among them, each memory module corresponds to a different filtering rule fragment.

2. The method according to claim 1, characterized in that The target hash value includes a query address, and the target hash value is subjected to data filtering processing according to the target filtering rule fragment corresponding to the target memory module to obtain a data filtering result, including: According to the query address, searching and obtaining a target bit value of an address bit corresponding to the query address in the target memory module; The data filtering result is determined according to the target bit value.

3. The method according to claim 2, characterized in that Determining the data filtering result according to the target bit value includes: If the target bit value is the first value, determining that the data filtering result is that the target hash value successfully matches the target filtering rule fragment; If the target bit value is the second value, it is determined that the data filtering result is that the target hash value fails to match the target filtering rule fragment, and the first value and the second value are different.

4. The method according to claim 1, characterized in that: The target hash value includes an index address, and determining a target memory module corresponding to the target hash value according to the target hash value includes: Determine, according to the index address, a memory module identifier corresponding to the index address; The target memory module corresponding to the memory module identifier is determined according to the memory module identifier.

5. The method according to claim 2, characterized in that: The method further comprises: Acquire a filtering rule set, and perform hash processing on the filtering rules included in the filtering rule set to obtain multiple rule hash values; Adjusting each bit value in the initial filtering rule table according to each rule hash value to obtain a target filtering rule table; According to the target filtering rule table, the bit value of the corresponding address bit in each of the memory modules is configured.

6. The method according to claim 1, characterized in that The step of performing data filtering processing on the target hash value according to the target filtering rule fragment corresponding to the target memory module to obtain a data filtering result includes: According to the target data filtering rule fragment corresponding to the target memory module, performing initial data filtering processing on the target hash value to obtain an initial data filtering result; If the initial data filtering result does not meet the preset condition, the initial data filtering result is continuously subjected to lower-level data filtering processing until a data filtering result meeting the preset condition is obtained, wherein the preset condition is related to a preset data filtering ratio.

7. A data filtering device, characterized in that: The device comprises: An acquisition module is used to acquire the data to be filtered and perform hash processing on the data to be filtered to obtain a target hash value; A filtering module, configured to determine a target memory module corresponding to the target hash value according to the target hash value, and perform data filtering processing on the target hash value according to a target filtering rule fragment corresponding to the target memory module to obtain a data filtering result; Among them, each memory module corresponds to a different filtering rule fragment.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images