Data screening method and device, equipment, storage medium and program product

Abstract

The invention relates to a data screening method and device, equipment, a storage medium and a program product, and the method comprises the steps: carrying out the rule matching according to a rule set corresponding to to-be-processed data and a preset rule management table, and generating a rule hit table; the rule set comprises at least one combination rule, the rule management table comprises indexes of sub-rules in all the combination rules, then determining a rule association table corresponding to the rule set according to the rule management table, and screening the to-be-processed data according to the rule hit table and the rule association table to obtain a screening result. According to the method, the rule relation between the combination rule and the sub-rules in the combination rule is maintained through the rule management table, the rule matching condition of the to-be-processed data is analyzed through the rule hit table and the rule association table, the matching effect of block rules can be achieved, and then the data screening efficiency is improved. In addition, according to the method, a plurality of sub-rules can be combined at will, and the purpose of more accurate data screening can be achieved.

Description

Technical Field

[0001] The present application relates to the technical field of network data processing, and in particular to a data screening method, device, equipment, storage medium and program product. Background Art

[0002] With the rapid development of network technology, network traffic is becoming increasingly complex, and the requirements for the accuracy of network traffic data packets are constantly increasing. Rule matching is a basic and very important technical means in the field of traffic detection. Due to its advantages of high efficiency, flexibility, and easy implementation, it is widely used in convergence and distribution, firewalls, deep packet inspection and other equipment. Therefore, in order to ensure the security and reliability of the network, it is very necessary to accurately filter data packets.

[0003] At present, the method for screening data messages is usually to perform matching screening in a regular multi-level serial manner. However, the above data message screening method has the problem of low screening efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a data screening method, device, equipment, storage medium and program product that can improve data screening efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a data screening method, the method comprising:

[0006] Matching rules according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes the indexes of sub-rules in all combination rules;

[0007] Determine the rule association table corresponding to the rule set according to the rule management table;

[0008] The data to be processed is screened according to the rule hit table and the rule association table to obtain the screening result.

[0009] The data screening method provided in the embodiment of the present application performs rule matching according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes the index of the sub-rules in all the combination rules, and then the rule association table corresponding to the rule set is determined according to the rule management table, and then the data to be processed is screened according to the rule hit table and the rule association table to obtain the screening result. The above method maintains the rule relationship between the combination rule and the sub-rules in the combination rule through the rule management table, and analyzes the rule matching of the data to be processed through the rule hit table and the rule association table, which can achieve the matching effect of the block rule, thereby improving the data screening efficiency. Moreover, the above method can arbitrarily combine multiple sub-rules to achieve the purpose of more accurate data screening.

[0010] In one embodiment, rule matching is performed according to a rule set corresponding to the data to be processed and a preset rule management table to generate a rule hit table, including:

[0011] Perform rule matching according to the rule set corresponding to the data to be processed to obtain the matching result;

[0012] Match the rules based on the hit results and the rule management table to generate a rule hit table.

[0013] In one embodiment, rule matching is performed according to the hit result and the rule management table to generate a rule hit table, including:

[0014] Determine the sub-rule of successful hit according to the hit result;

[0015] The index of the sub-rule is determined according to the rule management table, and the rule hit table is generated according to the index of the sub-rule.

[0016] In one embodiment, determining the index of the sub-rule according to the rule management table, and generating the rule hit table according to the index of the sub-rule, includes:

[0017] If the sub-rule is a non-public sub-rule, the index of the sub-rule is determined according to the rule management table, and a rule hit table is generated according to the index of the sub-rule;

[0018] If there are multiple sub-rules, and the multiple sub-rules include common sub-rules and non-common sub-rules, the index of the non-common sub-rule is determined according to the rule management table, and the index of the common sub-rule is determined according to the index of the non-common sub-rule and the rule management table, and the rule hit table is generated according to the index of the non-common sub-rule and the index of the common sub-rule.

[0019] The method described in the embodiment of the present application can flexibly process data by combining common sub-rules and non-common sub-rules, and can improve data processing efficiency by quickly determining the index of common sub-rules and the index of non-common sub-rules through the rule management table.

[0020] In one embodiment, the data to be processed is screened according to the rule hit table and the rule association table to obtain the screening results, including:

[0021] Determine whether the rule hit table is consistent with the rule association table;

[0022] If the rule hit table is consistent with the rule association table, then the screening result is determined to indicate that the data to be processed has passed the screening;

[0023] If the rule hit table is inconsistent with the rule association table, it is determined that the screening result indicates that the data to be processed fails the screening.

[0024] The method described in the embodiment of the present application can realize data screening and processing by comparing the rule hit table and the rule association table of the data to be processed. The method is simple and efficient and can improve the efficiency of data screening.

[0025] In one embodiment, the method further comprises:

[0026] Receive target sub-rules issued by users;

[0027] A rule management table is constructed according to the target sub-rule, the initial combination rule table and the initial common rule table; the rule management table includes the combination rule table and the common rule table.

[0028] In one embodiment, a rule management table is constructed according to the target sub-rule, the initial combination rule table and the initial common rule table, including:

[0029] Determine whether the target sub-rule is issued for the first time;

[0030] If the target sub-rule is issued for the first time, an address is assigned to the target sub-rule in the initial combination rule table, and an index of the target sub-rule is determined according to the address, and the initial combination rule table is updated according to the index of the target sub-rule, and a combination rule table is generated;

[0031] If the target sub-rule is not issued for the first time, when it is determined that the target sub-rule is a common sub-rule, the common sub-rule is stored in a preset area in the initial common rule table, and the combined rule table corresponding to the common sub-rule is bound to the preset area to generate a common rule table.

[0032] The method described in the embodiments of the present application can manage rules more efficiently and reduce waste of resources by distinguishing between the first issuance and non-first issuance of target sub-rules.

[0033] In a second aspect, the present application further provides a data screening device, the device comprising:

[0034] A generation module is used to perform rule matching according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes the indexes of the sub-rules in all the combination rules;

[0035] A determination module, used to determine a rule association table corresponding to a rule set according to a rule management table;

[0036] The screening module is used to screen the data to be processed according to the rule hit table and the rule association table to obtain the screening results.

[0037] In a third aspect, the present application further provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0038] Matching rules according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combined rule, and the rule management table includes indexes of sub-rules in all combined rules;

[0039] Determine the rule association table corresponding to the rule set according to the rule management table;

[0040] The data to be processed is screened according to the rule hit table and the rule association table to obtain the screening results.

[0041] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0042] Matching rules according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combined rule, and the rule management table includes indexes of sub-rules in all combined rules;

[0043] Determine the rule association table corresponding to the rule set according to the rule management table;

[0044] The data to be processed is screened according to the rule hit table and the rule association table to obtain the screening results.

[0045] In a fifth aspect, the present application further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0046] Matching rules according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes the indexes of sub-rules in all combination rules;

[0047] Determine the rule association table corresponding to the rule set according to the rule management table;

[0048] The data to be processed is screened according to the rule hit table and the rule association table to obtain the screening result.

[0049] The above-mentioned data screening method, device, equipment, storage medium and program product, the method performs rule matching according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes the index of the sub-rules in all the combination rules, and then the rule association table corresponding to the rule set is determined according to the rule management table, and then the data to be processed is screened according to the rule hit table and the rule association table to obtain the screening result. The above-mentioned method maintains the rule relationship between the combination rule and the sub-rules in the combination rule through the rule management table, and analyzes the rule matching of the data to be processed through the rule hit table and the rule association table, which can achieve the matching effect of the block rule, thereby improving the data screening efficiency. Moreover, the above-mentioned method can arbitrarily combine multiple sub-rules to achieve the purpose of more accurate data screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is an internal structure diagram of a computer device in one embodiment;

[0051] Figure 2 A schematic diagram of a flow chart of a data screening method in one embodiment;

[0052] Figure 3 A schematic diagram of the structure of a combination rule table in an embodiment;

[0053] Figure 4 A schematic diagram of the structure of a common rule table in one embodiment;

[0054] Figure 5 A schematic diagram of a flow chart of a data screening method in another embodiment;

[0055] Figure 6 A schematic diagram of a flow chart of a data screening method in another embodiment;

[0056] Figure 7 A schematic diagram of a flow chart of a data screening method in another embodiment;

[0057] Figure 8 A schematic diagram of a flow chart of a data screening method in another embodiment;

[0058] Fig. 9 A schematic diagram of a flow chart of a data screening method in another embodiment;

[0059] Fig.10 A schematic diagram of the structure of an initial combination rule table in an embodiment;

[0060] Fig.11 A schematic diagram of the structure of an initial public rule table according to an embodiment;

[0061] Fig.12 A schematic diagram of a flow chart of a data screening method in another embodiment;

[0062] Fig.13 A schematic diagram of a flow chart of a data screening method in another embodiment;

[0063] Fig.14 FIG. 4 is a structural block diagram of a data screening device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] With the rapid development of network technology, network traffic is becoming increasingly complex, and the requirements for the screening accuracy of network traffic data packets are constantly increasing. Rule matching is a basic and very important technical means in the field of traffic detection. Due to its advantages of high efficiency, flexibility, and easy implementation, it is widely used in devices such as convergence and diversion, firewalls, and deep packet inspection. Therefore, in order to ensure the security and reliability of the network, it is very necessary to accurately screen data packets. At present, the method for screening data packets is usually to match and screen through a multi-level serial method of rules, but the above-mentioned data packet screening method has the problem of low screening efficiency. The present application provides a data screening method, which aims to solve the above-mentioned technical problems. The following embodiments will specifically illustrate the data screening method described in the present application.

[0066] The data screening method provided in the embodiment of the present application can be applied to Figure 1 In the computer device shown in the figure, the computer device can be a terminal or a server, and its internal structure diagram can be as shown in FIG. Figure 1As shown, the computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data screening method is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0067] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0068] In one embodiment, Figure 2 As shown, a data screening method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:

[0069] S201, performing rule matching according to a rule set corresponding to the data to be processed and a preset rule management table to generate a rule hit table.

[0070] The data to be processed may be traffic message data. The rule set includes at least one combination rule, each combination rule includes multiple sub-rules, and the types of sub-rules include public sub-rules and non-public sub-rules. Public sub-rules are public rules in multiple combination rules, and non-public sub-rules are private rules in combination rules. For example, combination rule A is abc, and combination rule B is abf, then sub-rules a and b are public sub-rules, and sub-rules c and f are non-public sub-rules. The rule management table is created based on multiple preset sub-rules, including a combination rule table and a public rule table. Figure 3 As shown, the combination rule table includes multiple combination rules (see ABC in the figure), each combination rule includes the indexes of multiple sub-rules (see abc in the figure), the index of the sub-rule is represented by a bit mark, and the index of each sub-rule is associated with the address of the sub-rule (see [0][1][2] in the figure) and the content of the sub-rule. When the bit mark is 1, it means that the address has a sub-rule, and when the bit mark is 0, it means that the address does not have a sub-rule. Figure 4 As shown, the common rule table is an index table of common sub-rules, including multiple common sub-rules (see abc in the figure), and each common sub-rule is associated with multiple combination rule tables containing the common sub-rule (see AB in the figure). For example, combination rule A is abc, combination rule B is adf, and the common sub-rule is a.

[0071] In the embodiment of the present application, the computer device can create a rule management table in advance according to the large number of sub-rules formulated by the user. Specifically, a combined rule table can be created according to non-public sub-rules, and a public rule table can be created according to public sub-rules and non-public sub-rules to obtain a preset rule management table containing a combined rule table and a public rule table, and the rule management table is stored under a preset path. After the computer device receives the data to be processed, the rule set corresponding to the data to be processed can be determined, and the rule management table under the preset path can be called, and then the rule matching is performed on the data to be processed according to the rule set and the rule management table to generate a rule hit table. Specifically, the corresponding rule set can be determined in advance according to the type of the data to be processed, and a mapping relationship between the type of the data to be processed and the rule set is formed. After the computer device receives the data to be processed, the type of the data to be processed can be determined, and then the rule set is determined according to the mapping relationship between the type of the data to be processed and the rule set. Optionally, the rule set can be pre-set as a default rule set. Optionally, a user interface can be set, and a selection control of the rule set can be selected in response to a selection control triggered by a user. The computer device can pop up a list of combined rules, and generate a rule set in response to one or more combined rules selected by the user. Optionally, the rule set may be carried by a data packet corresponding to the data to be processed. Optionally, the computer device may receive the data to be processed forwarded by other devices. Optionally, the computer device may obtain the data to be processed input by the user on the user interface from the user interface.

[0072] S202: Determine a rule association table corresponding to the rule set according to the rule management table.

[0073] The rule association table is determined by the index of the sub-rule in the rule management table.

[0074] In the embodiment of the present application, after determining the rule set, the computer device can further determine the combination rule in the rule set, then determine each sub-rule in the combination rule, and finally query the index of each sub-rule in the rule management table to generate a rule association table. For example, the rule set includes AB, the combination rule A is [abc], and the combination rule B is [adef], then the rule association table 1 is [111000], and the rule association table 2 is [111100].

[0075] S203, screening the data to be processed according to the rule hit table and the rule association table to obtain screening results.

[0076] The screening result includes whether the data to be processed passes the screening or whether the data to be processed fails the screening.

[0077] In an embodiment of the present application, after the computer device obtains the rule hit table and the rule association table based on the above steps, it can compare the rule hit table and the rule association table to obtain a comparison result, and then determine a screening result based on the comparison result.

[0078] The data screening method provided in the embodiment of the present application performs rule matching according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes the index of the sub-rules in all the combination rules, and then the rule association table corresponding to the rule set is determined according to the rule management table, and then the data to be processed is screened according to the rule hit table and the rule association table to obtain the screening result. The above method maintains the rule relationship between the combination rule and the sub-rules in the combination rule through the rule management table, and analyzes the rule matching of the data to be processed through the rule hit table and the rule association table, which can achieve the matching effect of the block rule, thereby improving the data screening efficiency. Moreover, the above method can arbitrarily combine multiple sub-rules to achieve the purpose of more accurate data screening.

[0079] In one embodiment, a specific implementation method for generating a rule hit table according to a rule set and a rule management table is also provided. Figure 5 As shown, the above step S201 of "matching rules according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table" includes:

[0080] S301, performing rule matching according to a rule set corresponding to the data to be processed to obtain a matching result.

[0081] The hit result includes the hit sub-rule.

[0082] In the embodiment of the present application, after the computer device determines the rule, it can use each combination rule in the rule set to perform rule matching on the data to be processed to obtain a matching result. Optionally, each sub-rule in each combination rule in the rule set can be used to perform rule matching to obtain a matching result.

[0083] S302: Perform rule matching according to the hit result and the rule management table to generate a rule hit table.

[0084] In the embodiment of the present application, after the computer device obtains the hit result and the rule management table based on the above steps, it can determine the sub-rule that has successfully hit, and then match the rule hit table with the sub-rule that has successfully hit and the rule management table.

[0085] Specifically, Figure 6 As shown, the above step S302 of "matching rules according to the hit results and the rule management table to generate a rule hit table" includes:

[0086] S401, determining the sub-rules that have successfully hit according to the hit result.

[0087] In the embodiment of the present application, after obtaining the hit result, the computer device can determine the sub-rule that hits successfully according to the hit result.

[0088] S402: Determine the index of the sub-rule according to the rule management table, and generate a rule hit table according to the index of the sub-rule.

[0089] In an embodiment of the present application, after the computer device obtains a successfully hit sub-rule based on the above steps, it can query the sub-rule index corresponding to the hit sub-rule in the rule management table and generate a rule hit table according to the sub-rule index.

[0090] Furthermore, the above sub-rules that successfully hit may be non-public sub-rules, or may be non-public sub-rules and public sub-rules, such as Figure 7 As shown, the above step S402 of "determining the index of the sub-rule according to the rule management table, and generating the rule hit table according to the index of the sub-rule" includes:

[0091] S501: If the sub-rule is a non-public sub-rule, determine the index of the sub-rule according to the rule management table, and generate a rule hit table according to the index of the sub-rule.

[0092] In the embodiment of the present application, after the computer device determines the sub-rule that has hit successfully, it can further determine the type of the sub-rule that has hit successfully. If the sub-rule that has hit successfully is one or more non-public sub-rules, then the index of the sub-rule that has hit successfully can be determined according to the combined rule table in the rule management table, and the rule hit table can be generated according to the index of the sub-rule. For example, if the combined rule M1 is abc, then the index of abc in the combined rule table is a[0], b[1], c[2], then the rule association table is [111000], if the data to be processed hits the sub-rule abc, then the rule hit table is [111000], if the data to be processed hits the sub-rule ac, then the rule hit table is [101000] in the same way.

[0093] S502, if the sub-rule consists of multiple sub-rules, and the multiple sub-rules include common sub-rules and non-common sub-rules, the index of the non-common sub-rule is determined according to the rule management table, and the index of the common sub-rule is determined according to the index of the non-common sub-rule and the rule management table, and the rule hit table is generated according to the index of the non-common sub-rule and the index of the common sub-rule.

[0094] In an embodiment of the present application, if the successfully hit sub-rule includes a common sub-rule and a non-common sub-rule, then the index of the non-common sub-rule can be determined first according to the combined rule table in the rule management table, and then the index of the common sub-rule can be determined according to the index of the non-common sub-rule and the common rule table in the rule management table, and finally the rule hit table is generated according to the index of the non-common sub-rule and the index of the common sub-rule. For example, combined rule 1 is 1bc and combined rule 2 is fe1g. Among them, rule 1 is a common sub-rule. Similarly, based on the above method, it can be determined that the association table of M1 is M1: [111000], and the association table of M2 is M2: [111100]. The position of the common sub-rule 1 in the combined rule table is 1[0], and the position in the common rule table is 1[2]. If the successfully hit sub-rule is 1bcf, then the index of the non-common sub-rule can be determined first according to the combined rule table. The combined rule table can determine that the position of b in M1 is b[1], and the position of c in M1 is is c[2], and the position of f in M2 is f[0]. The number of rule hit tables is the same as that of association tables, so according to the non-public sub-rules, it can be preliminarily determined that the rule hit table m1 is [011000] and m2 is [100000]. For the public sub-rule 1, the public rule table can determine that the position of 1 in M1 is 1[0] and the position in M2 is 1[2]. Therefore, by further updating m1 and m2, it can be determined that the rule hit table m1 is [111000] and m2 is [101000].

[0095] The method described in the embodiment of the present application can flexibly process data by combining common sub-rules and non-common sub-rules, and can improve data processing efficiency by quickly determining the index of common sub-rules and the index of non-common sub-rules through the rule management table.

[0096] In one embodiment, a specific implementation method for filtering the data to be processed is also provided, such as Figure 8 As shown, the above step S203 of "screening the data to be processed according to the rule hit table and the rule association table to obtain the screening result" includes:

[0097] S601, determining whether the rule hit table is consistent with the rule association table.

[0098] In the embodiment of the present application, after obtaining the rule hit table and the rule association table, the computer device can compare each bit in the rule hit table with the corresponding bit in the rule association table to determine whether the rule hit table is consistent with the rule association table. For example, compare each element in the rule association table M1 [111000] with each element in the rule hit table m1 [011000] to determine whether they are consistent.

[0099] S602: If the rule hit table is consistent with the rule association table, the screening result indicates that the data to be processed has passed the screening.

[0100] In the embodiment of the present application, if the computer device determines based on the above steps that the rule hit table is consistent with the rule association table, the screening result indicates that the data to be processed has passed the screening.

[0101] S603: If the rule hit table is inconsistent with the rule association table, the screening result indicates that the data to be processed fails the screening.

[0102] In the embodiment of the present application, if the computer device determines based on the above steps that the rule hit table and the rule association table are inconsistent, the screening result indicates that the data to be processed does not pass the screening.

[0103] The method described in the embodiment of the present application can realize data screening and processing by comparing the rule hit table and the rule association table of the data to be processed. The method is simple and efficient and can improve the efficiency of data screening.

[0104] In one embodiment, Fig. 9 As shown, the above data processing method also includes:

[0105] S701, receiving a target sub-rule sent by a user.

[0106] Among them, the target sub-rule is the rule set by the user.

[0107] In the embodiment of the present application, the computer device may receive the target sub-rule input by the user in the user interface. Optionally, the computer device may also receive the target sub-rule uploaded by the user in a preset path.

[0108] S702: construct a rule management table according to the target sub-rule, the initial combination rule table and the initial common rule table. The rule management table includes a combination rule table and a common rule table.

[0109] Among them, Fig.10 As shown, the initial combination rule table includes a combination rule ID, a combination rule address, a non-public sub-rule ID, a non-public sub-rule address, and a non-public sub-rule identification bit. Fig.11 As shown, the initial public rule table includes a public sub-rule ID, a public sub-rule address, a combination rule ID, a combination rule address, a combination rule identification bit, a non-public sub-rule ID, a non-public sub-rule address and a non-public sub-rule identification bit.

[0110] In an embodiment of the present application, an initial combination rule table and an initial public rule table can be created in advance, and then after the computer device obtains the target sub-rule based on the above steps, the initial combination rule table and the initial public rule table can be updated according to the target sub-rule to obtain a rule management table.

[0111] Specifically, Fig.12 As shown, the "building a rule management table according to the target sub-rule, the initial combination rule table and the initial common rule table" in the above step S702 includes:

[0112] S801, determining whether the target sub-rule is issued for the first time.

[0113] In the embodiment of the present application, after the computer device has updated the initial combination rule table and the initial public rule table with the sub-rule each time in the history, the position of the sub-rule in the combination rule table, the position of the sub-rule in the public rule table, and the content of the sub-rule can be synchronously updated in the preset hash table. After the computer device obtains the target sub-rule based on the above steps, it can search for the target sub-rule in the preset hash table, and determine whether the target sub-rule is issued for the first time based on the search result. If the target sub-rule cannot be found in the preset hash table, it is determined that the target sub-rule is issued for the first time. If the target sub-rule can be found in the preset hash table, it is determined that the target sub-rule is issued for the first time.

[0114] S802, if the target sub-rule is issued for the first time, an address is allocated to the target sub-rule in the initial combination rule table, and an index of the target sub-rule is determined according to the address, and the initial combination rule table is updated according to the index of the target sub-rule, and a combination rule table is generated.

[0115] In an embodiment of the present application, the computer device determines based on the above steps that the target sub-rule is issued for the first time, then an address can be assigned to the target sub-rule in the initial combination rule table, and the index of the target sub-rule can be determined according to the address, and the initial combination rule table can be updated according to the index of the target sub-rule, and a combination rule table can be generated.

[0116] S803, if the target sub-rule is not issued for the first time, then when it is determined that the target sub-rule is a common sub-rule, the common sub-rule is stored in a preset area in the initial common rule table, and the combined rule table corresponding to the common sub-rule is bound to the preset area to generate a common rule table.

[0117] In an embodiment of the present application, the computer device determines based on the above steps that the target sub-rule is not issued for the first time. In this case, it can be determined that the target sub-rule is a common sub-rule. Then, the common sub-rule can be stored in a preset area in the initial common rule table, and the combined rule table corresponding to the common sub-rule can be bound to the preset area to generate a common rule table.

[0118] The method described in the embodiments of the present application can manage rules more efficiently and reduce waste of resources by distinguishing between the first issuance and non-first issuance of target sub-rules.

[0119] Based on all the above embodiments, a data screening method is also provided. Fig.13As shown, the method includes:

[0120] S901, receiving a target sub-rule sent by a user.

[0121] S902: Determine whether the target sub-rule is issued for the first time.

[0122] S903, if the target sub-rule is issued for the first time, allocate an address for the target sub-rule in the initial combination rule table, determine the index of the target sub-rule according to the address, update the initial combination rule table according to the index of the target sub-rule, and generate a combination rule table.

[0123] S904, if the target sub-rule is not issued for the first time, then when it is determined that the target sub-rule is a common sub-rule, the common sub-rule is stored in a preset area in the initial common rule table, and the combined rule table corresponding to the common sub-rule is bound to the preset area to generate a common rule table.

[0124] S905, performing rule matching according to the rule set corresponding to the data to be processed to obtain matching results.

[0125] S906, determining the sub-rules that have successfully hit according to the hit result.

[0126] S907: If the sub-rule is a non-public sub-rule, determine the index of the sub-rule according to the rule management table, and generate a rule hit table according to the index of the sub-rule.

[0127] S908: If the sub-rule consists of multiple sub-rules, and the multiple sub-rules include common sub-rules and non-common sub-rules, determine the index of the non-common sub-rule according to the rule management table.

[0128] S909: Determine the index of the common sub-rule according to the index of the non-common sub-rule and the rule management table.

[0129] S910: Generate a rule hit table according to the indexes of the non-common sub-rules and the indexes of the common sub-rules, wherein the rule set includes at least one combined rule, and the rule management table includes the indexes of the sub-rules in all combined rules.

[0130] S911, determining a rule association table corresponding to the rule set according to the rule management table.

[0131] S912: Determine whether the rule hit table is consistent with the rule association table.

[0132] S913, if the rule hit table is consistent with the rule association table, the screening result indicates that the data to be processed has passed the screening.

[0133] S914, if the rule hit table is inconsistent with the rule association table, the screening result indicates that the data to be processed fails the screening.

[0134] The method described in the embodiment of the present application is to associate sub-rules with the same combination rule ID into one combination rule; generate a combination rule association table indexed by the rule ID, and assign a sub-index position in the combination rule association table to each sub-rule, and distinguish between non-public sub-rules and public sub-rules. Match all rule types in parallel for data, obtain the combination rule ID and position information in the group in the sub-rule after a single sub-rule hits, and generate a combination rule hit table; fill all sub-rule information in the message match hit into the combination rule hit table; when the generated combination hit table is the same as the combination rule association table, the combination rule is considered to have matched successfully; this solution is not limited by the rule type and the number of rules, and can flexibly add and delete combinations. The rules are highly accurate, and one device can achieve the function without increasing costs.

[0135] The methods described in the above steps are all described in the above embodiments. Please refer to the above description for details and will not be repeated here.

[0136] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0137] Based on the same inventive concept, the embodiment of the present application also provides a data screening device for implementing the data screening method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data screening device embodiments provided below can refer to the limitations on the data screening method above, and will not be repeated here.

[0138] In one embodiment, Fig.14 As shown, a data screening device is provided, comprising:

[0139] The generating module 11 is used to perform rule matching according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combined rule, and the rule management table includes the indexes of the sub-rules in all combined rules.

[0140] The determination module 12 is used to determine the rule association table corresponding to the rule set according to the rule management table.

[0141] The screening module 13 is used to screen the data to be processed according to the rule hit table and the rule association table to obtain a screening result.

[0142] In one embodiment, the generation module includes:

[0143] The hit unit is used to perform rule hits according to the rule set corresponding to the data to be processed to obtain a hit result.

[0144] The matching unit is used to perform rule matching according to the hit result and the rule management table to generate a rule hit table.

[0145] In one embodiment, the matching unit includes:

[0146] The determination sub-unit is used to determine the sub-rule that hits successfully according to the hit result.

[0147] The generating sub-unit is used to determine the index of the sub-rule according to the rule management table, and generate the rule hit table according to the index of the sub-rule.

[0148] In one embodiment, the above-mentioned generation sub-unit is specifically used to determine the index of the sub-rule according to the rule management table if the sub-rule is a non-common sub-rule, and generate a rule hit table according to the index of the sub-rule; if the sub-rule is multiple sub-rules, and the multiple sub-rules include common sub-rules and non-common sub-rules, then determine the index of the non-common sub-rule according to the rule management table, and determine the index of the common sub-rule according to the index of the non-common sub-rule and the rule management table, and generate a rule hit table according to the index of the non-common sub-rule and the index of the common sub-rule.

[0149] In one embodiment, the screening module comprises:

[0150] The first determining unit is used to determine whether the rule hit table is consistent with the rule association table.

[0151] The second determining unit is used for, if the rule hit table is consistent with the rule association table, the screening result indicates that the data to be processed has passed the screening.

[0152] The third determining unit is used for, if the rule hit table is inconsistent with the rule association table, the screening result indicates that the data to be processed fails the screening.

[0153] In one embodiment, the data screening device further includes:

[0154] The receiving module is used to receive the target sub-rules sent by the user.

[0155] The construction module is used to construct a rule management table according to the target sub-rule, the initial combination rule table and the initial common rule table; the rule management table includes a combination rule table and a common rule table.

[0156] In one embodiment, the above-mentioned building blocks include:

[0157] The fourth determining unit is used to determine whether the target sub-rule is issued for the first time.

[0158] The first generating unit is used to assign an address to the target sub-rule in the initial combination rule table if the target sub-rule is issued for the first time, determine the index of the target sub-rule according to the address, update the initial combination rule table according to the index of the target sub-rule, and generate a combination rule table.

[0159] The second generating unit is used for storing the common sub-rule into a preset area in the initial common rule table if the target sub-rule is not issued for the first time and binding the combined rule table corresponding to the common sub-rule with the preset area to generate a common rule table.

[0160] Each module in the above data screening device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0161] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0162] Matching rules according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes the indexes of sub-rules in all combination rules;

[0163] Determine the rule association table corresponding to the rule set according to the rule management table;

[0164] The data to be processed is screened according to the rule hit table and the rule association table to obtain the screening result.

[0165] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0166] Perform rule matching according to the rule set corresponding to the data to be processed to obtain the matching result;

[0167] Match the rules based on the hit results and the rule management table to generate a rule hit table.

[0168] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0169] Determine the sub-rule of successful hit according to the hit result;

[0170] The index of the sub-rule is determined according to the rule management table, and the rule hit table is generated according to the index of the sub-rule.

[0171] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0172] If the sub-rule is a non-public sub-rule, the index of the sub-rule is determined according to the rule management table, and a rule hit table is generated according to the index of the sub-rule;

[0173] If there are multiple sub-rules, and the multiple sub-rules include common sub-rules and non-common sub-rules, the index of the non-common sub-rule is determined according to the rule management table, and the index of the common sub-rule is determined according to the index of the non-common sub-rule and the rule management table, and the rule hit table is generated according to the index of the non-common sub-rule and the index of the common sub-rule.

[0174] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0175] Determine whether the rule hit table is consistent with the rule association table;

[0176] If the rule hit table is consistent with the rule association table, then the screening result is determined to indicate that the data to be processed has passed the screening;

[0177] If the rule hit table is inconsistent with the rule association table, it is determined that the screening result indicates that the data to be processed fails the screening.

[0178] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0179] Receive target sub-rules issued by users;

[0180] A rule management table is constructed according to the target sub-rule, the initial combination rule table and the initial common rule table; the rule management table includes the combination rule table and the common rule table.

[0181] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0182] Determine whether the target sub-rule is issued for the first time;

[0183] If the target sub-rule is issued for the first time, an address is assigned to the target sub-rule in the initial combination rule table, and an index of the target sub-rule is determined according to the address, and the initial combination rule table is updated according to the index of the target sub-rule, and a combination rule table is generated;

[0184] If the target sub-rule is not issued for the first time, when it is determined that the target sub-rule is a common sub-rule, the common sub-rule is stored in a preset area in the initial common rule table, and the combined rule table corresponding to the common sub-rule is bound to the preset area to generate a common rule table.

[0185] The computer device provided in the above embodiment has an implementation principle and technical effects similar to those of the above method embodiment, which will not be described in detail here.

[0186] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0187] Matching rules according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes the indexes of sub-rules in all combination rules;

[0188] Determine the rule association table corresponding to the rule set according to the rule management table;

[0189] The data to be processed is screened according to the rule hit table and the rule association table to obtain the screening result.

[0190] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0191] Perform rule matching according to the rule set corresponding to the data to be processed to obtain the matching result;

[0192] Match the rules based on the hit results and the rule management table to generate a rule hit table.

[0193] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0194] Determine the sub-rule of successful hit according to the hit result;

[0195] The index of the sub-rule is determined according to the rule management table, and the rule hit table is generated according to the index of the sub-rule.

[0196] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0197] If the sub-rule is a non-public sub-rule, the index of the sub-rule is determined according to the rule management table, and a rule hit table is generated according to the index of the sub-rule;

[0198] If there are multiple sub-rules, and the multiple sub-rules include common sub-rules and non-common sub-rules, the index of the non-common sub-rule is determined according to the rule management table, and the index of the common sub-rule is determined according to the index of the non-common sub-rule and the rule management table, and the rule hit table is generated according to the index of the non-common sub-rule and the index of the common sub-rule.

[0199] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0200] Determine whether the rule hit table is consistent with the rule association table;

[0201] If the rule hit table is consistent with the rule association table, then the screening result is determined to indicate that the data to be processed has passed the screening;

[0202] If the rule hit table is inconsistent with the rule association table, it is determined that the screening result indicates that the data to be processed fails the screening.

[0203] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0204] Receive target sub-rules issued by users;

[0205] A rule management table is constructed according to the target sub-rule, the initial combination rule table and the initial common rule table; the rule management table includes the combination rule table and the common rule table.

[0206] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0207] Determine whether the target sub-rule is issued for the first time;

[0208] If the target sub-rule is issued for the first time, an address is assigned to the target sub-rule in the initial combination rule table, and an index of the target sub-rule is determined according to the address, and the initial combination rule table is updated according to the index of the target sub-rule, and a combination rule table is generated;

[0209] If the target sub-rule is not issued for the first time, when it is determined that the target sub-rule is a common sub-rule, the common sub-rule is stored in a preset area in the initial common rule table, and the combined rule table corresponding to the common sub-rule is bound to the preset area to generate a common rule table.

[0210] The above embodiment provides a computer-readable storage medium, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.

[0211] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0212] Matching rules according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes the indexes of sub-rules in all combination rules;

[0213] Determine the rule association table corresponding to the rule set according to the rule management table;

[0214] The data to be processed is screened according to the rule hit table and the rule association table to obtain the screening result.

[0215] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0216] Perform rule matching according to the rule set corresponding to the data to be processed to obtain the matching result;

[0217] Match the rules based on the hit results and the rule management table to generate a rule hit table.

[0218] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0219] Determine the sub-rule of successful hit according to the hit result;

[0220] The index of the sub-rule is determined according to the rule management table, and the rule hit table is generated according to the index of the sub-rule.

[0221] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0222] If the sub-rule is a non-public sub-rule, the index of the sub-rule is determined according to the rule management table, and a rule hit table is generated according to the index of the sub-rule;

[0223] If there are multiple sub-rules, and the multiple sub-rules include common sub-rules and non-common sub-rules, the index of the non-common sub-rule is determined according to the rule management table, and the index of the common sub-rule is determined according to the index of the non-common sub-rule and the rule management table, and the rule hit table is generated according to the index of the non-common sub-rule and the index of the common sub-rule.

[0224] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0225] Determine whether the rule hit table is consistent with the rule association table;

[0226] If the rule hit table is consistent with the rule association table, then the screening result is determined to indicate that the data to be processed has passed the screening;

[0227] If the rule hit table is inconsistent with the rule association table, it is determined that the screening result indicates that the data to be processed fails the screening.

[0228] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0229] Receive target sub-rules issued by users;

[0230] A rule management table is constructed according to the target sub-rule, the initial combination rule table and the initial common rule table; the rule management table includes the combination rule table and the common rule table.

[0231] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0232] Determine whether the target sub-rule is issued for the first time;

[0233] If the target sub-rule is issued for the first time, an address is assigned to the target sub-rule in the initial combination rule table, and an index of the target sub-rule is determined according to the address, and the initial combination rule table is updated according to the index of the target sub-rule, and a combination rule table is generated;

[0234] If the target sub-rule is not issued for the first time, when it is determined that the target sub-rule is a common sub-rule, the common sub-rule is stored in a preset area in the initial common rule table, and the combined rule table corresponding to the common sub-rule is bound to the preset area to generate a common rule table.

[0235] The above embodiment provides a computer program product, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.

[0236] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0237] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0238] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data screening method, characterized in that: The method comprises: Perform rule matching according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes indexes of sub-rules in all the combination rules; Determine a rule association table corresponding to the rule set according to the rule management table; The data to be processed is screened according to the rule hit table and the rule association table to obtain a screening result.

2. The method according to claim 1, characterized in that The rule matching is performed according to the rule set corresponding to the data to be processed and the preset rule management table to generate a rule hit table, including: Perform rule matching according to the rule set corresponding to the data to be processed to obtain a matching result; Rule matching is performed according to the hit result and the rule management table to generate the rule hit table.

3. The method according to claim 2, characterized in that The step of performing rule matching according to the hit result and the rule management table to generate the rule hit table includes: Determine a sub-rule of successful hit according to the hit result; The index of the sub-rule is determined according to the rule management table, and the rule hit table is generated according to the index of the sub-rule.

4. The method according to claim 3, characterized in that: The determining the index of the sub-rule according to the rule management table, and generating the rule hit table according to the index of the sub-rule, includes: If the sub-rule is a non-public sub-rule, determining the index of the sub-rule according to the rule management table, and generating the rule hit table according to the index of the sub-rule; If the sub-rule consists of multiple sub-rules, and the multiple sub-rules include common sub-rules and non-common sub-rules, the index of the non-common sub-rule is determined according to the rule management table, and the index of the common sub-rule is determined according to the index of the non-common sub-rule and the rule management table, and the rule hit table is generated according to the index of the non-common sub-rule and the index of the common sub-rule.

5. The method according to claim 1, characterized in that: The step of screening the data to be processed according to the rule hit table and the rule association table to obtain screening results includes: Determine whether the rule hit table is consistent with the rule association table; If the rule hit table is consistent with the rule association table, determining that the screening result indicates that the data to be processed has passed the screening; If the rule hit table is inconsistent with the rule association table, it is determined that the screening result indicates that the data to be processed fails the screening.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Receive target sub-rules issued by users; The rule management table is constructed according to the target sub-rule, the initial combination rule table and the initial common rule table; the rule management table includes a combination rule table and a common rule table.

7. The method according to claim 6, characterized in that The step of constructing the rule management table according to the target sub-rule, the initial combination rule table and the initial common rule table includes: Determine whether the target sub-rule is issued for the first time; If the target sub-rule is issued for the first time, an address is assigned to the target sub-rule in the initial combination rule table, and an index of the target sub-rule is determined according to the address, and the initial combination rule table is updated according to the index of the target sub-rule, and the combination rule table is generated; If the target sub-rule is not issued for the first time, then when it is determined that the target sub-rule is a common sub-rule, the common sub-rule is stored in a preset area in the initial common rule table, and the combined rule table corresponding to the common sub-rule is bound to the preset area to generate the common rule table.

8. A data screening device, characterized in that: The device comprises: A generation module, used to perform rule matching according to a rule set corresponding to the data to be processed and a preset rule management table, and generate a rule hit table; the rule set includes at least one combination rule, and the rule management table includes indexes of sub-rules in all the combination rules; A determination module, used to determine a rule association table corresponding to the rule set according to the rule management table; The screening module is used to screen the data to be processed according to the rule hit table and the rule association table to obtain a screening result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images