A method and apparatus for processing multiple data streams with equivalent index
By constructing an equivalence index expression graph and determining equivalence expression paths, the problem of low efficiency in network security analysis and processing of multi-data stream networks is solved, enabling rapid security risk discovery and handling.
Patent Information
- Application Number
- CN202310140906.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-15
- Publication Date
- 2025-12-16
- Estimated Expiration
- 2043-02-15
AI Technical Summary
Existing technologies for network security analysis and processing of multiple data streams are inefficient, making it difficult to quickly identify and address potential security risks.
By constructing an equivalence index expression graph, we use association analysis expressions to determine the chain of equivalence indexable expressions, traverse the equivalence index expression graph to determine the equivalence expression path, and use it as an index for flow table analysis and processing.
It improves the efficiency of multi-data stream analysis and processing, enabling the rapid detection and handling of potential security risks.
Smart Images

Figure CN116303596B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer processing, in particular to a multi-data stream processing method and device with equivalent index. In addition, it also relates to an electronic device, a non-transitory computer-readable storage medium and a computer program product. BACKGROUND
[0002] In recent years, with the rapid development of network technology, various network attacks are becoming more and more common. With the continuous increase of data flow, the realization of multi-data stream network security analysis and processing brings great challenges to people.
[0003] At present, when performing network security analysis and processing on multi-data streams, it is usually necessary to traverse all data streams to find target data streams. The multi-data stream searching efficiency and network security analysis efficiency of this method are low, which makes it difficult to quickly process potential various security risks. Therefore, how to design a more efficient multi-data stream security processing method has become a difficult problem to be solved. SUMMARY
[0004] Therefore, the present application provides a multi-data stream processing method and device with equivalent index to solve the defects of the prior art that the multi-data stream processing scheme is low in efficiency, which makes it difficult to quickly find and process potential various abnormal data.
[0005] The present application provides a multi-data stream processing method with equivalent index, comprising:
[0006] According to the association analysis expression, the corresponding equivalent indexable expression chain is determined; the association analysis expression is used to represent the association relationship between the parameters of the data flow stored in each flow table; the equivalent indexable expression chain includes at least two equivalent indexable expressions, and the values of each equivalent indexable expression are the same;
[0007] Based on the equivalent indexable expression chain, an equivalent index expression graph is constructed; the equivalent index expression graph includes multiple graph nodes and edges between the graph nodes; the graph nodes include the equivalent indexable expressions corresponding to the flow tables, and the edges include the association relationship between the equivalent indexable expressions corresponding to the two connected graph nodes;
[0008] The graph nodes in the equivalent index expression graph are traversed to determine the equivalent expression path of the equivalent index expression graph;
[0009] The equivalent expression path is taken as the index corresponding to the flow table, and the data flow is analyzed and processed based on the index of the flow table.
[0010] Further, the equivalent index expression graph is constructed based on the equivalent indexable expression chain, specifically comprising:
[0011] traversing the chain of equivalent indexable expressions to obtain an equivalent indexable expression;
[0012] determining a first expression of a first type of the equivalent indexable expression and a graph node of a first type corresponding to the first expression as a first element in the equivalent index expression graph; the first element is used to indicate a storage location of information of a flow table corresponding to the first expression;
[0013] determining a second expression of a second type of the equivalent indexable expression and a graph node of a second type corresponding to the second expression as a second element in the equivalent index expression graph; the second element is used to indicate a storage location of information of a flow table corresponding to the second expression;
[0014] obtaining the equivalent index expression graph according to the first element and the second element;
[0015] wherein the graph node comprises a first identifier of a flow table and an equivalent indexable expression corresponding to the first identifier.
[0016] Further, the traversing of the graph nodes in the equivalent index expression graph to determine an equivalent expression path of the equivalent index expression graph comprises:
[0017] traversing all graph nodes in the equivalent index expression graph to determine at least one path composed of graph nodes and corresponding edges;
[0018] determining a path including all flow tables in the at least one path as the equivalent expression path.
[0019] Further, the determining of the corresponding equivalent indexable expression chain according to the associated analysis expression comprises:
[0020] transforming the stored data flows into a logical tree based on the associated relationship between the parameters of the stored data flows in each flow table;
[0021] traversing nodes in the logical tree to determine a first node used for performing characteristic calculation;
[0022] determining whether a first child node of the first node and left and right child nodes of the first child node constitute an indexable expression; the two flow table identifiers included in the indexable expression are different;
[0023] add the expression corresponding to the left child node, the identifier of the flow table corresponding to the left child node, the expression corresponding to the right child node, and the identifier of the flow table corresponding to the right child node to an initial expression chain; the initial expression chain is a preset associated structure expression for storing expressions and identifiers;
[0024] In a case where all nodes in the logical tree are traversed, the initial expression chain is determined as the equivalent indexable expression chain.
[0025] Further, the determination of whether the first child node of the first node and the left child node and the right child node of the first child node constitute an indexable expression includes:
[0026] In a case where the first child node is determined to be a child node representing an equal operation, it is determined whether the left child node and all child nodes related to the left child node are first single-table field expression nodes, and whether the right child node and all child nodes related to the right child node are second single-table field expression nodes;
[0027] In a case where the left child node and all child nodes related to the left child node are determined to be the first single-table field expression nodes, and the right child node and all child nodes related to the right child node are determined to be the second single-table field expression nodes, it is determined, based on the identifier of the flow table corresponding to the first single-table field expression node and the identifier of the flow table corresponding to the second single-table field expression node, whether the first child node of the first node and the left child node and the right child node of the first child node constitute an indexable expression.
[0028] Further, the determination, based on the identifier of the flow table corresponding to the first single-table field expression node and the identifier of the flow table corresponding to the second single-table field expression node, of whether the first child node of the first node and the left child node and the right child node of the first child node constitute an indexable expression includes:
[0029] In a case where the left child node and all child nodes related to the left child node are determined to be the first single-table field expression nodes, the first number of times that the first single-table field expression refers to a first table field is updated based on the identifier of the flow table corresponding to the first single-table field expression node;
[0030] In a case where the right child node and all child nodes related to the right child node are determined to be the second single-table field expression nodes, the second number of times that the second single-table field expression refers to a second table field is updated based on the identifier of the flow table corresponding to the second single-table field expression node.
[0031] In a case where it is determined that the first number of times is greater than zero, the second number of times is greater than zero, and the identifier of the flow table corresponding to the first single-table field expression node is different from the identifier of the flow table corresponding to the second single-table field expression node, it is determined that the first node, the first child node of the first node, the left child node of the first child node, and the right child node of the first child node form an indexable expression.
[0032] Further, the analysis and processing of the data flow based on the index of the flow table specifically include:
[0033] Obtaining the flow table identifier corresponding to the data flow to be analyzed and the index value corresponding to the data flow to be analyzed;
[0034] Based on the index value corresponding to the data flow to be analyzed, the index of the flow table corresponding to the flow table identifier, the data flow association analysis is performed to obtain a corresponding data set; based on the preset network security analysis and processing flow and the data set, the data flow to be analyzed is processed.
[0035] Further, the processing of the data flow to be analyzed based on the preset network security analysis and processing flow and the data set specifically includes:
[0036] Based on a preset deletion processing flow, the data flow to be deleted associated with the index is deleted from the flow table corresponding to the flow table identifier in the index table slot, and the data flow to be deleted includes a timeout data flow; or, based on a preset insertion processing flow, the data flow to be inserted associated with the index is added to a preset hit table; or, based on a preset update processing flow, the data flow to be updated associated with the index is updated; the data set includes the data flow to be deleted, the data flow to be inserted, or the data flow to be updated; the hit table stores data associated with successful association analysis.
[0037] The application further provides a multi-data flow processing device with equivalent indexes, comprising:
[0038] An equivalent indexable expression chain determination unit is configured to determine a corresponding equivalent indexable expression chain according to an association analysis expression; the association analysis expression is used to represent the association relationship between the parameters of the data flow stored in each flow table; the equivalent indexable expression chain includes at least two equivalent indexable expressions, and the values of each equivalent indexable expression are the same.
[0039] An equivalent index expression graph construction unit is configured to construct an equivalent index expression graph based on the equivalent indexable expression chain; the equivalent index expression graph comprises a plurality of graph nodes and edges between the graph nodes; the graph nodes comprise equivalent indexable expressions corresponding to flow tables, and the edges comprise association relationships of equivalent indexable expressions corresponding to two connected graph nodes;
[0040] An equivalent expression path determination unit is configured to traverse the graph nodes in the equivalent index expression graph and determine an equivalent expression path of the equivalent index expression graph.
[0041] A data analysis processing unit is configured to use the equivalent expression path as an index corresponding to a flow table, and analyze and process the data stream based on the index of the flow table.
[0042] Further, the equivalent index expression graph construction unit is specifically configured to:
[0043] Traverse the equivalent indexable expression chain to obtain an equivalent indexable expression;
[0044] Determine a first expression of a first type of the equivalent indexable expression and a graph node of a first type corresponding to the first expression as a first element in the equivalent index expression graph; the first element is used to indicate a storage location of information of a flow table corresponding to the first expression.
[0045] Determine a second expression of a second type of the equivalent indexable expression and a graph node of a second type corresponding to the second expression as a second element in the equivalent index expression graph; the second element is used to indicate a storage location of information of a flow table corresponding to the second expression.
[0046] Obtain the equivalent index expression graph according to the first element and the second element.
[0047] The graph node comprises a first identifier of a flow table and an equivalent indexable expression corresponding to the first identifier.
[0048] Further, the equivalent expression path determination unit is specifically configured to:
[0049] Traverse all graph nodes in the equivalent index expression graph to determine at least one path composed of a graph node and a corresponding edge;
[0050] Determine a path comprising all flow tables in the at least one path as the equivalent expression path.
[0051] Further, the equivalent indexable expression chain determination unit is specifically configured to:
[0052] transform the stored data flows into a logical tree based on a correlation between parameters of the data flows stored in each flow table;
[0053] traverse nodes in the logical tree to determine a first node for performing a characterization calculation;
[0054] determine whether the first child node of the first node and the left child node and the right child node of the first child node form an indexable expression; the indexable expression includes flow table identifiers of two flow tables that are different;
[0055] in a case where it is determined that the first child node of the first node, the left child node and the right child node of the first child node form the indexable expression, add an expression corresponding to the left child node, an identifier of a flow table corresponding to the left child node, an expression corresponding to the right child node, and an identifier of a flow table corresponding to the right child node to an initial expression chain; the initial expression chain is a preset association structure expression for storing expressions and identifiers;
[0056] in a case where all nodes in the logical tree are traversed, determine the initial expression chain as the equivalent indexable expression chain.
[0057] Further, the determination of whether the first child node of the first node and the left child node and the right child node of the first child node form an indexable expression includes:
[0058] in a case where it is determined that the first child node is a child node representing an equal operation, determine whether the left child node and all child nodes related to the left child node are first single-table field expression nodes, and determine whether the right child node and all child nodes related to the right child node are second single-table field expression nodes;
[0059] in a case where it is determined that the left child node and all child nodes related to the left child node are the first single-table field expression nodes, and the right child node and all child nodes related to the right child node are the second single-table field expression nodes, determine whether the first child node of the first node and the left child node and the right child node of the first child node form an indexable expression based on an identifier of a flow table corresponding to the first single-table field expression node and an identifier of a flow table corresponding to the second single-table field expression node.
[0060] Further, the determination of whether the first child node of the first node and the left child node and the right child node of the first child node form an indexable expression based on the identifier of the flow table corresponding to the first single-table field expression node and the identifier of the flow table corresponding to the second single-table field expression node includes:
[0061] in a case where it is determined that the left child node and all child nodes related to the left child node are the first single table field expression node, updating a first number of times that the first single table field expression references a first table field based on an identifier of a flow table corresponding to the first single table field expression node;
[0062] in a case where it is determined that the right child node and all child nodes related to the right child node are the second single table field expression node, updating a second number of times that the second single table field expression references a second table field based on an identifier of a flow table corresponding to the second single table field expression node;
[0063] in a case where it is determined that the first number of times is greater than zero, the second number of times is greater than zero, and the identifier of the flow table corresponding to the first single table field expression node is different from the identifier of the flow table corresponding to the second single table field expression node, determining that the first child node of the first node and the left child node and the right child node of the first child node form an indexable expression.
[0064] Further, the analyzing and processing the data flow based on the index of the flow table specifically comprises:
[0065] obtaining an index value corresponding to the data flow to be analyzed and an identifier of a flow table corresponding to the data flow to be analyzed;
[0066] performing data flow correlation analysis based on the index value corresponding to the data flow to be analyzed and the index of the flow table corresponding to the flow table identifier to obtain a corresponding data set; and processing the data flow to be analyzed based on a preset network security analysis and processing procedure and the data set.
[0067] Further, the processing the data flow to be analyzed based on the preset network security analysis and processing procedure and the data set specifically comprises:
[0068] based on a preset deletion processing procedure, deleting a data flow to be deleted associated with the index from the flow table corresponding to the flow table identifier in the index table slot, the data flow to be deleted including a timeout data flow; or, based on a preset insertion processing procedure, adding a data flow to be inserted associated with the index to a preset hit table; or, based on a preset update processing procedure, performing update processing on a data flow to be updated associated with the index; the data set includes the data flow to be deleted, the data flow to be inserted or the data flow to be updated; the hit table stores data related to successful association analysis.
[0069] The application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for processing multiple data flows with an equivalent index as described above when executing the program.
[0070] The application further provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the multi-data stream processing method with equivalent index as any one of the above.
[0071] The application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the multi-data stream processing method with equivalent index as any one of the above.
[0072] The multi-data stream processing method with equivalent index provided by the application determines the corresponding equivalent indexable expression chain through the associated analysis expression, the equivalent indexable expression chain comprises at least two equivalent indexable expressions, and the values of each equivalent indexable expression are the same; the equivalent index expression graph is constructed based on the equivalent indexable expression chain, the equivalent expression path of the equivalent index expression graph is determined by traversing the graph nodes in the equivalent index expression graph, the equivalent expression path is taken as the index corresponding to the flow table, the data stream is analyzed and processed based on the index of the flow table, the method can perform equivalent index analysis on the associated analysis expression of the multi-data stream, so that the associated analysis of the multi-data stream is quickly realized, and the efficiency of the multi-data stream analysis and processing is improved. BRIEF DESCRIPTION OF DRAWINGS
[0073] In order to more clearly illustrate the technical solutions of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.
[0074] Figure 1 is a flow diagram of the multi-data stream processing method with equivalent index provided by the application;
[0075] Figure 2 is a mapping graph of the equivalent indexable expression chain provided by the application;
[0076] Figure 3 is a mapping graph of the equivalent indexable expression provided by the application;
[0077] Figure 4 is a flow diagram of the index analysis of the associated analysis expression provided by the application;
[0078] Figure 5 is a flow diagram of the search key equivalent indexable expression chain provided by the application;
[0079] Figure 6 is a flow diagram of the indexable expression analyzer provided by the application;
[0080] Figure 7 is a flow chart of a single table field expression analyzer provided by the present application;
[0081] Figure 8 is a flow chart of a build indexable expression graph provided by the present application;
[0082] Figure 9 is a flow chart of an analyze indexable expression graph provided by the present application;
[0083] Figure 10 is a flow chart of an analyze GraphNode provided by the present application;
[0084] Figure 11 is a flow chart of an insert data stream data provided by the present application;
[0085] Figure 12 is a flow chart of a build join computation table slot provided by the present application;
[0086] Figure 13 is a flow chart of a join table data provided by the present application;
[0087] Figure 14 is a flow chart of a delete data stream data provided by the present application;
[0088] Figure 15 is a flow chart of an update data stream data provided by the present application;
[0089] Figure 16 is a flow chart of a notify data stream data delete provided by the present application;
[0090] Figure 17 is a structural schematic diagram of a multi-data stream processing device with an equal value index provided by the present application;
[0091] Figure 18 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION
[0092] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.
[0093] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work belong to the protection scope of the present application.
[0094] It should be noted that the terms "first", "second" and the like used in the present application are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.
[0095] The embodiments of the multi-data stream processing method with equivalent index based on the present application will be described in detail below. As shown in Figure 1 The flowchart of the multi-data stream processing method with equivalent index provided by the embodiments of the present application is shown in the figure, and the specific process includes the following steps:
[0096] Step 101: According to the correlation analysis expression, determine the corresponding equivalent indexable expression chain; the correlation analysis expression is used to represent the correlation between the parameters of the data stream stored in each flow table; the equivalent indexable expression chain includes at least two equivalent indexable expressions, and the value of each equivalent indexable expression is the same.
[0097] In the embodiment of the present application, the associated analysis expression input by the user or the associated analysis expression stored in advance can be acquired in advance before the step is performed. The associated relationship between the parameters of the data flow stored in each flow table is determined according to the associated analysis expression, and then the corresponding equivalent indexable expression chain is obtained. The data flow is the network behavior data acquired from the network device. The structure definition of the associated analysis expression includes the flow table definition and the associated expression definition. The associated expression is a Boolean expression, and the result of the associated expression is true or false. For example, table1.sip==table2.sip and table2.sport<table3.sport, the semantic of the associated analysis expression is that when the source IP of the flow table table1 is equal to the sip of the flow table table2 and the source port of the flow table table2 is less than the flow table table3, the result of the associated analysis expression is true, otherwise, it is false. Wherein, the associated analysis expression can obtain the index expression by using the index construction, the number of flow table data to be associated each time is greatly reduced through the index expression, thereby improving the performance of the associated analysis, and the index expression can include the single table field expression, the indexable expression, the key Boolean expression, etc. The equivalent indexable expression chain can refer to an example of mathematical equivalent transmission: given A==B, B==C, then A==C can be derived, and the definition of the equivalent indexable expression chain is that the expression chain composed of multiple [indexable expressions], and the indexable expression can be derived into the mutually equal expression through the equivalent condition. For example, table1.sip==table2.sip and table2.sip==table3.dip and table3.dip==table4.sip. In the above example, there are three [indexable expressions] at all times: table1.sip==table2.sip, table2.sip==table3.dip, and table3.dip==table4.sip, which are connected through the two equivalent conditions of table2.sip and table3.dip, and can constitute an equivalent chain => table1.sip==table2.sip==table3.dip==table4.sip, and here table1.sip==table2.sip and table2.sip==table3.dip and table3.dip==table4.sip constitutes an equivalent indexable expression chain.
[0098] Specifically, the structure of the correlation analysis expression can be defined as: join [flow table definition] if [correlation expression definition]. The structure of the correlation analysis expression includes two parts: a flow table and a correlation expression. The flow table is defined as [input stream name] as [flow table name], for example: stream1 as table1, stream2 as table2, stream3 as table3; wherein stream1, stream2 and stream3 correspond to the names of three different input data streams, and table1, table2 and table3 are the corresponding flow table names. The stream1 is defined as table1, the stream2 is defined as table2, and the stream3 is defined as table3. The correlation expression is a Boolean expression, and the expression result is true or false. For example: table1.sip == table2.sip and table2.sport < table3.sport, the expression semantics is that when the source ip of table table1 is equal to the sip of table table2 and the source port of table table2 is less than table table3, the expression is true, otherwise it is false. The complete correlation analysis expression is as follows: join stream1 as table1, stream2 as table2, stream3 as table3 if table1.sip == table2.sip and table2.sport < table3.sport. The correlation expression is the correlation analysis expression.
[0099] It should be noted that the correlation analysis expression is written by the user according to the analysis scene, which can have any writing method in theory. Among all the writing methods, there is a type of correlation expression, which can use the index construction method to greatly reduce the number of table data that needs to be correlated each time, thereby improving the performance of the correlation analysis. This special correlation expression includes single table field expression, indexable expression, key Boolean expression, key Boolean expression, etc., which are defined as follows:
[0100] The single table field expression contains table field expressions and only contains fields of the same flow table, that is, the expression only refers to the fields of one flow table, and there is no field reference of other tables in the expression. This case is defined as a single table field expression, for example: tablel.dport + tablel.sport == 80, where only the dport and sport fields of the flow table tablel are used, and thus it is a single table field expression. The following example is not a single table field expression: tablel.dport + table2.sport == 80, where the dport of the flow table tablel and the sport field of the flow table table2 are used, and thus it is not a single table field expression. The table field expression is composed of the flow table name, a character point. and a field name, such as the source IP field of the flow table tablel: tablel.sip. The indexable expression is defined as [single table field expression] == [single table field expression], where the single table field expressions on the left and the right correspond to different flow tables, that is, if the single table field expression on the left corresponds to the flow table tablel, then the single table field expression on the right cannot correspond to tablel, and vice versa. The key Boolean expression is defined as a Boolean expression if the value of the expression can determine the value of the entire associated expression, and it is a key Boolean expression. In the example join streaml as tablel, stream2 as table2, stream3 as table3 if tablel.sip == table2.sip and table2.sport < table3.sport, [tablel.sip == table2.sip] and [table2.sport < table3.sport] are both key Boolean expressions, because they can determine the value of the entire associated expression. A simpler key Boolean expression can be defined as a Boolean expression that can be directly connected by an and logical operation expression chain, for example: [key Boolean expressionl] and [key Boolean expression2] and [key Boolean expression3] or [non-key Boolean expression]. The indexable expression chain can refer to an example of mathematical equality transmission: given A == B, B == C, then it can be deduced that A == C. The definition of the indexable expression chain is that multiple [indexable expressions] constitute an expression chain, and the expressions can be deduced to be equal to each other by the equality condition, for example: tablel.sip == table2.sip and table2.sip == table3.dip and table3.dip ==
[0101] table4.sip In the above example, there are always three [indexable expressions]: tablel.sip == table2.sip, table2.sip == table3.dip, table3.dip == table4.sip, which are connected by two equal conditions table2.sip and table3.dip, and can form an equal chain =>
[0102] tablel.sip == table2.sip == table3.dip == table4.sip, where tablel.sip == table2.sip == table3.dip == table4.sip
[0103] table2.sip and table2.sip == table3.dip and table3.dip == table4.sip form an equal indexable expression chain. The indexable expressions that form the equal indexable expression chain are equal indexable expressions.
[0104] It should be further explained that whether an association analysis expression has an equal index depends on the following three conditions: contains [equal indexable expression chain]; all [indexable expressions] in this [equal indexable expression chain] are [key Boolean expressions] at the same time; this [equal indexable expression chain] contains all stream tables in [stream table definition]. An example of the expression is as follows:
[0105] join streaml as tablel, stream2 as table2, stream3 as table3 if tablel.sip == table2.sip and (tablel.sport < table2.sport or tablel.dip == table3.dip) and table2.sip == table3.dip;
[0106] In the above expression, tablel.sip == table2.sip and table2.sip == table3.dip are [indexable expressions] and are [key Boolean expressions]; tablel.dip == table3.dip is an [indexable expression], but is not a [key Boolean expression];
[0107] tablel.sip == table2.sip and table2.sip == table3.dip are [equal indexable expression chain] at the same time, in which table2.sip is an equal condition;
[0108] table1.sip==table2.sip and table2.sip==table3.dip contains table1, table2, table3, contains [stream table definition] stream1 as table1, stream2 as table2, stream3 as table3 all stream tables;
[0109] In summary, the index of the example join analysis expression is table1.sip==table2.sip and table2.sip==table3.dip.
[0110] table2.sip and table2.sip==table3.dip, and the index table (i.e., index) of table1, table2, table3 is constructed respectively using table1.sip, table2.sip, table3.dip.
[0111] If each [equality indexable expression] is regarded as an edge, the left value and the right value of the expression correspond to two nodes on the graph, and all [equality indexable expressions] in the [equality indexable expression chain] are mapped into the nodes and edges of the graph, then the expression index analysis method can be abstracted as how to find a path on the graph mapped by the [equality indexable expression chain], and the path contains all stream tables, and each stream table appears only once on the path. Still taking the above join analysis expression as an example: join stream1 as table1, stream2 as table2, stream3 as table3 if table1.sip==table2.sip and (table1.sport<table2.sport or table1.dip==table3.dip) and table2.sip==table3.dip, wherein table1.sip==table2.sip and table2.sip==table3.dip is an [equality indexable expression chain], and the mapped graph is as shown in Figure 2 .
[0112] The expression index analysis can be equivalent to finding a path in the graph that connects all the flow tables: tablel.sip<->table2.sip<->table3.dip. If such a path exists, it means that there is an equivalent index, and if not, there is no equivalent index. Referring to an example without an index: join streaml as tablel, stream2 as table2, stream3 as table3 if tablel.sip==table2.sip and (tablel.sport<table2.sport or tablel.dip==table3.dip) and table2.dip==table3.dip, where tablel.sip==table2.sip and table2.dip==table3.dip are [equivalent indexable expressions] in the expression, the mapping graph is as shown in Figure 3 In this example, there is no path that connects all the flow tables and contains tablel, table2 and table3, so the expression has no equivalent index.
[0113] In the implementation of the present step, first, the stored data streams are converted into corresponding logical trees based on the association between the parameters of the data streams stored in each flow table, that is, the association between the parameters of the data streams stored in each flow table is converted into a corresponding logical tree, then the nodes in the logical tree are traversed to determine a first node for representing an operation, and determine whether the first child node of the first node and the left and right child nodes of the first child node form an indexable expression. The two flow table identifiers included in the indexable expression are different. When it is determined that the first child node of the first node, the left and right child nodes of the first child node form the indexable expression, the expression corresponding to the left child node, the identifier of the flow table corresponding to the left child node, the expression corresponding to the right child node, and the identifier of the flow table corresponding to the right child node are added to an initial expression chain. When all the nodes in the logical tree are traversed, the initial expression chain is determined as an equivalent indexable expression chain. The corresponding specific process is as shown in Figure 5 The logical tree conversion is performed on the stored data, so that the corresponding data can be found from the stored data according to the expression. The node is a logical tree node. The first node is a logical tree root node, the first child node is an and node, and the left and right child nodes of the first child node are and logical tree left and right nodes, respectively.
[0114] The corresponding implementation process comprises: when it is determined that the first child node is a child node representing an equal operation, it is determined whether the left child node and all child nodes related to the left child node are first single-table field expression nodes, and it is determined whether the right child node and all child nodes related to the right child node are second single-table field expression nodes; when it is determined that the left child node and all child nodes related to the left child node are the first single-table field expression nodes, and the right child node and all child nodes related to the right child node are the second single-table field expression nodes, it is determined whether the first child node of the first node and the left child node and the right child node of the first child node constitute an indexable expression based on an identifier of a flow table corresponding to the first single-table field expression node and an identifier of a flow table corresponding to the second single-table field expression node. The corresponding specific process is shown in Figure 6 The child node representing the equal operation is an expression node. The left child node and the right child node are left and right child nodes of the current tree node. The first single-table field expression node is a [single-table field expression] node related to the left child node; and the second single-table field expression node is a [single-table field expression] node related to the right child node. Specifically, the first single-table field expression node can be a left node expression, and the identifier of the flow table corresponding to the first single-table field expression node can be a left node corresponding tableID (i.e., a flow table number index); the second single-table field expression node can be a right node expression, and the identifier of the flow table corresponding to the second single-table field expression node can be a right node corresponding tableID.
[0115] The first node, the left child node of the first child node, and the right child node of the first child node form an indexable expression based on the identifier of the flow table corresponding to the first single-table field expression node and the identifier of the flow table corresponding to the second single-table field expression node. The corresponding implementation process includes: when it is determined that the left child node and all child nodes related to the left child node are the first single-table field expression node, updating a first number of times that the first single-table field expression references a first table field based on the identifier of the flow table corresponding to the first single-table field expression node; when it is determined that the right child node and all child nodes related to the right child node are the second single-table field expression node, updating a second number of times that the second single-table field expression references a second table field based on the identifier of the flow table corresponding to the second single-table field expression node; when it is determined that the first number of times is greater than zero, the second number of times is greater than zero, and the identifier of the flow table corresponding to the first single-table field expression node is different from the identifier of the flow table corresponding to the second single-table field expression node, determining that the first node, the left child node of the first child node, and the right child node of the first child node form an indexable expression. For specific processes, refer to Figure 7 as shown.
[0116] Step 102: constructing an equivalent index expression graph based on the equivalent indexable expression chain; the equivalent index expression graph includes multiple graph nodes and edges between the graph nodes; the graph nodes include equivalent indexable expressions corresponding to flow tables, and the edges include the association relationship of the equivalent indexable expressions corresponding to the two connected graph nodes.
[0117] Specifically, by traversing the equivalent indexable expression chain, for each equivalent indexable expression, a first expression of a first type of the equivalent indexable expression and a graph node of a first type corresponding to the first expression are determined as a first element in the equivalent index expression graph. The first element is used to indicate the storage location of the information of the flow table corresponding to the first expression. A second expression of a second type of the equivalent indexable expression and a graph node of a second type corresponding to the second expression are determined as a second element in the equivalent index expression graph; the second element is used to indicate the storage location of the information of the flow table corresponding to the second expression, and the equivalent index expression graph is obtained.
[0118] As Figure 8The equal-index expression chain is shown in the last block of FIG. 1, and is, for example, table1.sip==table2.sip and table2.sip==table3.dip, where table1.sip==table2.sip and table2.sip==table3.dip are equal-index expression chains. The equal-index expression is, for example, table1.sip==table2.sip. The first expression of the first type represents a left node expression, for example, table1.sip in table1.sip==table2.sip. The graph node of the first type corresponding to the first expression corresponds to tableID in the left node expression. The equal-index expression graph is an array, and the first element is an element in the array. The second expression of the second type represents a right node expression, for example, table2.sip in table1.sip in table1.sip==table2.sip. The graph node of the second type corresponding to the second expression corresponds to tableID in the right node expression. As shown in FIG. 1, the equal-index expression graph is an array, and the first element is an element in the array. Figure 8 As shown in the last block of FIG. 1, after obtaining the equal-index expression graph, the method further includes: inserting the graph node of the first type into the list of graph nodes of the second type, and inserting the graph node of the second type into the list of graph nodes of the first type, that is, associating the left graph node with the right graph node.
[0119] Step 103: traversing the graph nodes in the equal-index expression graph to determine an equal expression path of the equal-index expression graph.
[0120] Specifically, at least one path composed of graph nodes and corresponding edges is determined by traversing all graph nodes in the equal-index expression graph, and then a path including all flow tables in the at least one path is determined as the equal expression path. That is, a method for specifically determining an equal expression path. The graph nodes include equal-index expressions corresponding to flow tables. The edges include association relationships of equal-index expressions corresponding to two connected graph nodes. The equal-index expression in the equal expression path is taken as an index, and the index table of the corresponding flow table is determined based on the index, so that only the data flow corresponding to the index in the flow table needs to be searched, without traversing all data, and the searching efficiency is high, and the efficiency of security analysis is further improved. The equal expression path includes graph nodes having the same identifier of flow tables.
[0121] Step 104: taking the equal expression path as an index of the flow table, and performing analysis and processing on the data flow based on the index of the flow table.
[0122] In the implementation process of the step, after the equivalent expression path is taken as the index corresponding to the flow table, the flow table identifier corresponding to the to-be-analyzed data flow and the index value corresponding to the to-be-analyzed data flow are obtained. Based on the index value corresponding to the to-be-analyzed data flow, the index of the flow table corresponding to the flow table identifier is subjected to data flow correlation analysis, and a corresponding data set is obtained; and based on a preset network security analysis and processing flow and the data set, the to-be-analyzed data flow is processed. Specifically, based on a preset deletion processing flow, the to-be-deleted data flow associated with the index is deleted from the flow table corresponding to the flow table identifier in the index table slot, and the to-be-deleted data flow includes a timeout data flow; or, based on a preset insertion processing flow, the to-be-inserted data flow associated with the index is added to a preset hit table; or, based on a preset update processing flow, the to-be-updated data flow associated with the index is subjected to update processing; and the data set includes the to-be-deleted data flow, the to-be-inserted data flow, or the to-be-updated data flow. The index is an equivalent index. The equivalent index is an index expression condition used for constructing a flow table, that is, the left value and the right value of the expression are equal. For a multi-flow-table equivalent index, an equivalent expression chain (i.e., an equivalent indexable expression chain) is required, which includes all flow tables. For example, join stream1 as table1, stream2 as table2, stream3 as table3 if table1.sip == table2.sip and table2.sip == table3.dip. In this example, the expressions table1.sip == table2.sip and table2.sip == table3.dip have an equivalent transmission expression table2.sip, and through the equivalent transmission expression, an equivalent expression chain table1.sip == table2.sip == table3.dip including all flow tables can be constructed. Therefore, table1.sip, table2.sip, and table3.dip can be used for table1, table2, and table3 to construct an equivalent index. Through this principle, an index (i.e., an index value) can be constructed on each flow table, which can greatly improve the performance of flow table correlation calculation.It should be noted that the real-time multi-data stream association analysis parameter with equal value index is defined as follows: the parameter N is defined as the number of flow tables, N >= 3; the parameter streamClassifier is defined as a data stream classifier, which labels the input data stream as a corresponding flow table serial number; the parameter keyedTableSlot[N] is defined as an index table slot, tableSlot[n] corresponds to each flow table respectively, 0 <= n < N; the parameter keyBuilderSlot[N] is defined as an index value generator slot, keyBuilderSlot[n] corresponds to an index value generator of each flow table respectively, 0 <= n < N; the parameter matchedTable is defined as a hit table, which saves table data of successful association, and one row in the table corresponds to one pair of successfully associated data; the parameter joinFilter is an association filter, which calculates a group of data (the number of data streams is N) that needs to be associated, and if the value is true, it indicates that the association is successful, and if the value is false, it indicates that the association fails. The application determines the index table of the corresponding flow table based on the index by taking the equal value indexable expression as the index, only needs to find the data stream corresponding to the index in the flow table, traverses part of the data in the flow table based on the equal value index, does not need to traverse all the data, and has high data searching efficiency, thereby improving the efficiency of security analysis.
[0123] In the embodiment of the application, the real-time multi-data stream association analysis method with equal value index comprises four processes, and the specific processes are as follows: a data stream data insertion process insertStreamData(data), the process content of which is to insert corresponding data stream data <data>; delete stream data flow deleteStreamData(data), whose flow content is to delete the corresponding data stream data <data>; update the data stream data, specifically including two steps: 1. delete the data stream data <olddata>;2. insert stream data <newdata>; notify data stream data timeout procedure onStreamDataExpire(data) whose procedure content is receiving data stream data <data>timeout notification, and delete the corresponding data stream data after receiving the notification <data>The insertStreamData(data) includes two key processes: (1) a buildJoinTableSlot process, which is called in the form of buildJoinTableSlot(tableIndex, keyValue), where tableIndex is a table index, and the key in the keyValue is a current index value (i.e., an index), and the buildJoinTableSlot process returns a joinTableSlot. A joinTableRows process, which is called in the form of joinTableRows(joinTableSlot, joinRowSlot, tableIndex), is a process content of the joinTableRows data. Specifically, the joinTableSlot is a joinTableSlot, the joinRowSlot is a joinRowSlot, and the tableIndex is a table index.
[0124] As shown in Figure 11 : it is an insertStreamData process, i.e., an insert processing process. Specifically: (1) the insertStreamData(data) is called, and the input data stream data is <data>; (2) data flow classifier <streamclassifier>Computing <data>corresponding flow table number index; (3) input <data>, using the index value generator slot keyBuilderSlot[index] to calculate the index value keyValue; (4) calling the build join table slot flow process, and setting the join table slot joinTableSlot = buildJoinTableSlot(index, keyValue) (see the build join table slot buildJoinTableSlot flow process below); (4) defining the join calculation data slot joinRowSlot[N], the data type is an array, defining the table sequence number tableIndex, and setting tableIndex = 0; (6) calling joinTableRows(joinTableSlot, joinRowSlot, tableIndex) (see the join table data joinTableRows flow process below) to perform table data join calculation; (7) setting the table sequence number tableIndex = tableIndex + 1; (8) if tableIndex < joinTableSlot.tableCount, returning to step (6) to continue the join calculation; (9) if tableIndex = joinTableSlot.tableCount, returning to step (2) to continue the join calculation. <keyvalue>For the index value, the following is used <data>Store to table slot
[0125] <keyedtableslot>The Middle <index>in the keyedTableSlot[index] ; (8) to <data>Delete the registration of the listener.
[0126] As Figure 12 shown, it is a process of building a join computing table slot buildJoinTableSlot. Specifically: (1) call buildJoinTableSlot(index, keyValue), input parameter <index, keyValue>; (2) define joinTableSlot[N] in the join computing table slot, the data type is array, set all initial values of the join computing table in the join computing table slot as empty table, and set the joinTableSlot[N] as the input parameter <index, keyValue>. <data>(3) defining index n = 0, 0 <= n < N; (4) judging condition n < N, if yes, executing step (5), if not, ending the flow and returning <jointableslot>; (5) judgment condition n == 0 <index>if yes, then step (4) is performed, if not, step (6) is performed; (6) using the index value <keyvalue>In the index table slot corresponding to the index table keyedTableSlot[n], the index value corresponding data set is queried, and the queried data set is set as keyedRows, and if keyedRows is empty, the process is ended and an empty correlation calculation table slot is returned <jointableslot>If keyedRows is not empty, then perform step (7); (7) create a dataset <keyedrows>The joinTableRows is added to the corresponding joinTableSlot[n] in the join computation table slot, and n = n + 1 is set, and after completion, step (4) is executed.
[0127] As shown in Figure 13 , the joinTableRows flow of the associated table data. Specifically: (1) call joinTableRows(joinTableSlot, joinRowSlot, tableIndex), input parameters <joinTableSlot, joinRowSlot, tableIndex>; (2) define the current table currentTable, and set currentTable = joinTableSlot[tableIndex]; (3) traverse the table <currenttable>For each row of data <row>performing the calculation; (4)
[0128] <currenttable>Is there data in the table? <row>if yes, go to step 5, if no, go to step (9); (5) set joinRowSlot[tableIndex] = rowSlot[tableIndex] + 1 <row>and determine whether tableIndex + 1 is equal to N, if equal to N, then execute step (6), if not equal to N, then execute step (8); (6) set the associated calculation data slot <joinrowslot> <joinfilter>correlation calculation is performed if <joinfilter>If the result of the calculation is true, step (7) is performed, and if false, step (4) is performed; (7) insert <joinRowSlot[0], joinRowSlot[l],..., joinRowSlot[N-l]> into the hit table <matchedtable>, step (4) is executed after the completion; (8) recursively call joinTableRows(joinTableSlot, joinRowSlot, tableIndex+1), step (4) is executed after the completion of the call; (9) the current function call returns.
[0129] As shown in Figure 14 , it is a delete data stream data deleteStreamData(data) flow, that is, a deletion process. Specifically: (1) call deleteStreamData(data), input the data stream data to be deleted <data>(2) cancel the sending of <data>A registered deletion listener. (3) From <matchedtable>Query if there are associated hit rows, and if so, delete the corresponding hit data. (4) Dataflow classifier <streamclassifier>Computing <data>corresponding table number index. (5) input <data>keyValue is calculated by using the keyBuilderSlot[index] in the index value generator slot.(6) The index value (i.e., index) <keyvalue>From the index table slot <keyedtableslot>The Middle <index>deletion from the flow table keyedTableSlot[index] of the flow table (i.e., flow table identifier or flow table number) <data>corresponding data. (7) If the index table slot <keyedtableslot>If all the flow tables <keyedTableSlot[0], the flow table corresponding to the index table, keyedTableSlot[1],..., keyedTableSlot[N-1]> are empty, then delete the current computing unit.
[0130] As shown in Figure 15 Fig. 6, it is a flow of updating data stream data updateStreamData(oldData, newData), i.e. an updating process. Specifically: (1) call updateStreamData(oldData, newData), input the data stream data to be updated <oldData, newData>.(2) call the flow of deleting data stream data deleteStreamData(oldData) to delete the data stream data <oldData>.(3) call the flow of adding data stream data addStreamData(newData) to add the data stream data <newData>.(4) return the result of the updating process. <olddata>(3) Call the insertStreamData procedure insertStreamData(NewData) to insert <newdata>.
[0131] As Figure 16 shown, it is a notification data stream data timeout onStreamDataExpire(data) process. Specifically: (1) call onStreamExpire(data), input the timeout data stream data <data>(2) from <matchedtable>Query if there are associated hit rows, and if so, delete the corresponding hit data. (3) Data flow classifier <streamclassifier>Computing <data>corresponding table number index. (4) input <data>Use the keyBuilderSlot[index] in the index value generator slot to calculate the index value keyValue. (5) Use the index value (i.e., the index). <keyvalue>From the index table slot <keyedtableslot>The Middle <index>deletion from the flow table keyedTableSlot[index] (i.e., flow table encoding or flow table identification) <data>corresponding data. (6) If the index table slot <keyedtableslot>If all the flow tables <keyedTableSlot[0], keyedTableSlot[1], …, keyedTableSlot[N-1]> are empty, the current corresponding calculation unit is deleted.
[0132] It should be noted that in the embodiment of the present application, the index analysis method global parameter is defined as follows: N represents the number of flow tables, N >= 3. The index analysis includes three processes: (1) search key equal index expression chain searchEqualIndexExpression, the process content is called in the form of searchEqualIndexExpression (expression), wherein expression is the associated analysis expression to be analyzed; the process return value is all the key equal index expression chains searched equalIndexExpressionList, equalIndexExpressionList is a chain table, which stores expressions meeting the conditions of [key Boolean expression] and [indexable expression]. (2) build index expression graph buildIndexExpressionGraph, the process content is called in the form of buildIndexExpressionGraph (equalIndexExpressionList), wherein equalIndexExpressionList is the key equal index expression chain searched in the process searchEqualIndexExpression; the process return value is specifically the index expression graph indexExpressionGraph. (3) in the actual implementation process, analyze the index expression graph analyseIndexExpressionGraph, specifically, the process content is called in the form of analyseIndexExpressionGraph (indexExpressionGraph), wherein indexExpressionGraph is the index expression graph built in the process buildIndexExpressionGraph; if the index is found, the process return value is the final index, and if no index is found, no index is returned.
[0133] In the embodiment of the present application, as Figure 4 The shown is a real-time multi-data stream correlation analysis expression index analysis method. (1) Input correlation analysis expression join [stream table definition] if [correlation expression definition]. (2) Call search key equal indexable expression chain process to search all expressions in the expression that meet the [key Boolean expression] and [indexable expression] conditions, and set equalIndexExpressionList = searchEqualIndexExpression (correlation analysis expression). (3) Call build indexable expression graph process to build an indexable expression graph, and correspondingly, set indexExpressionGraph = buildIndexExpressionGraph (equalIndexExpressionList). (4) Call analyze indexable expression graph process to analyze the indexable expression path tree analyseIndexExpressionGraph (indexExpressionGraph), and return the result after completion.
[0134]
[0135] As shown in Figure 5 , search key equal indexable expression searchEqualIndexExpression. Wherein, the process parameter equalIndexExpressionList is defined as a table, and stores all expressions that meet the [key Boolean expression] and [indexable expression] conditions. Specifically, (1) input correlation analysis expression join [stream table definition] if [correlation expression definition] (2) parse the correlation analysis expression into a correlation analysis expression logic tree, and locate the root node of the logic tree where the [correlation expression] is located. (3) Traverse all nodes in the logic tree from the root node of the [correlation expression] logic tree. (4) Whether there are still nodes of the logic tree that have not been traversed, if yes, execute step (5), if no, execute step (8). (5) Judge whether the current logic tree node is an and node, if yes, execute step (6), if no, execute step (7). (6) Call [indexable expression] analyzer (see [indexable expression] analyzer part below) to analyze the and logic tree left node (i.e. left child node) and right node (i.e. right child node), and execute step (4) after completion. (7) Skip the analysis of the current logic tree node, and execute step (4). (8) End the process, and return equalIndexExpressionList.
[0136] As shown in Figure 6 The shown: it is [indexable expression] analyzer. (1) input the logical tree node to be analyzed. (2) judge whether the current logical tree node is a == expression node, if yes, execute step (3), if not, end the analysis process. (3) call [single table field expression] analyzer (for details, see the [single table field expression] analyzer part below) A to analyze the left node (i.e. the left child node) of the node (i.e. the first child node), call [single table field expression] analyzer (for details, see the [single table field expression] analyzer part below) B to analyze the right node (i.e. the right child node) of the node (i.e. the first child node), further judge whether it conforms to the variable value of analyzer A <tableid>greater than or equal to 0 and the variable value of analyzer B <tableid>greater than or equal to 0 and variable values of two analyzers A and B <tableid>The values are not equal, if it is consistent, then step 4 is executed, if it is not consistent, then the analysis process is ended. (4) The current expression corresponding value <(left node expression, left node corresponding table ID), (right node expression, right node corresponding table ID)> is added to the linked list <equalindexexpressionlist>and end the process.
[0137] As Figure 7 shown: It is a [single table field expression] analyzer. Specifically, the [single table field expression] analyzer parameter tableFieldCount[N] is defined as a table field reference count record table, which records the number of times the corresponding table field is referenced in the current expression. Among them, tableFieldCount[0] records the reference count of the flow table field numbered 0, and so on. tableFieldCount[n] records the reference count of the flow table field numbered n, where 0 <= n < N; the initial value of tableFieldCount[n] is set to 0. The [single table field expression] analyzer parameter tableID is defined as the flow table identifier, and the default value is -! If in the embodiment, for all tableFieldCount[n] in tableFieldCount[N], where 0 <= n < N, there is exactly one tableFieldCount[n] > 0, then set tableID = n; otherwise, set tableID = -1. Specifically, (1) Input the logic tree node to be analyzed. (2) Traverse all logic tree nodes starting from the logic tree node to be analyzed. (3) Whether there are still logic tree nodes that have not been traversed. If so, execute step (4); if not, execute step (7). (4) Whether the current logic tree node is a [single table field expression] node (i.e., the first single table field expression node). If so, execute step 5; if not, execute step (6). (5) Obtain the flow table number index corresponding to the current field table, set tableFieldCount[index] += 1, and after completion, execute step (3). (6) Continue to analyze the left and right child nodes of the current tree node (i.e., the left child node and the right child node of the first child node), and after completion, execute step (3). (7) If in the embodiment, for all tableFieldCount[n] in tableFieldCount[N], where 0 <= n < N, there is exactly one tableFieldCount[n] > 0, then set tableID = n; otherwise, set tableID = -1. After completion, end the process. The flow table is the data table
[0138] Figure 8 <equalindexexpressionlist>(2) define an index expression graph tree indexExpressionGraph[N], where each indexExpressionGraph[n], 0 <= n < N, is structured as a table indexed by expression, with value as data structure LinkNode. (3) traverse the key equivalent index expression linked list <equalindexexpressionlist>(4) Key-Value Indexed Expression List
[0139] <equalindexexpressionlist>If there is still data not traversed, execute step (5), if not, end the flow and return the indexable expression graph indexExpressionGraph. (5) For the current traversed data <(left node expression, left node corresponding tableID), (right node expression, right node corresponding tableID)>, take <left node expression> as the index value of the flow table, get the left node GraphNode (tableID = left node corresponding tableID, expression = left node expression) from indexExpressionGraph [<left node corresponding tableID>], if not, create in the table, set as leftLinkNode; take <right node expression> as the index value of the flow table, get the right node GraphNode (tableID = right node corresponding tableID, expression = right node expression) from indexExpressionGraph [<right node corresponding tableID>], if not, create in the flow table, set as rightLinkNode. (6) Insert rightLinkNode into the list of leftLinkNode.linkNodes, and insert leftLinkNode into the list of rightLinkNode.linkNodes, and execute step (4) after completion.
[0140] As Figure 9 shown: it is analyseIndexExpressionGraph. In which the flow parameter currentIndexExpressionGraphPath[N] is defined as an array, which saves the current equivalent expression path, for all currentIndexExpressionGraphPath[n] initial value is set to empty, 0 <= n <= N. Flow parameter pathNodeCount is defined as the number of nodes in the equivalent expression path, used to record how many nodes have been in currentIndexExpressionGraphPath, when pathNodeCount == N, it means that an equivalent expression path is found. Flow parameter indexExpressionGraphPaths is defined as the set of equivalent expression paths, which saves all the current equivalent expression paths that meet the requirements. The key flow is analyseGraphNode, the flow content is called in the form of analyseGraphNode(node), where node is the node to be analyzed. Specifically, (1) the input parameter is the indexable expression path tree <indexExpressionGraph[N]>.(2) In the embodiment of the present application, each GraphNode in the indexable expression graph is traversed, that is, all GraphNode nodes in indexExpressionGraph[n], 0 <= n <= N.(3) Is there any GraphNode node in indexExpressionGraph[N] that has not been traversed? If yes, step (4) is executed, if no, step (5) is executed.(4) Set the current GraphNode node as node, call the sub-flow analyseGraphNode(node) (see the analyseGraphNode section below), and execute step (3) after completion.(5) If indexExpressionGraphPaths is not empty, return
[0141] The first equivalent expression path in the indexExpressionGraphPaths list is used as an index, and if it is empty, there is no index, and no index is returned.
[0142] As Figure 10 As shown: This is the analysis of GraphNode nodes. Specifically, the process of analyzing GraphNode nodes includes: (1) Input parameter GraphNode node node. (2) Determine whether currentIndexExpressionGraphPath[node.tableID] is empty. If it is empty, proceed to step (3). If it is not empty, exit the process. (3) In the embodiment, set currentIndexExpressionGraphPath[node.tableID] = node, set pathNodeCount = pathNodeCount + 1, and after completion, determine whether pathNodeCount is equal to N. If it is equal, proceed to step (4). If it is not equal, proceed to step (5). (4) Transfer the currentIndexExpressionGraphPath[N] to N.
[0143] <currentIndexExpressionGraphPath[0],
[0144] currentIndexExpressionGraphPath[1],…,
[0145] The currentIndexExpressionGraphPath[n]>(0<=n<=N) is treated as an equality expression path and saved to indexExpressionGraphPaths. After completion, step (6) is executed. (5) Iterate through node.linkNodes and recursively call the process analyzeGraphNode(linkNode) for each GraphNode in linkNodes. After completion, step (6) is executed. (6) Set currentIndexExpressionGraphPath[node.tableID] to null and set pathNodeCount = pathNodeCount□1. After completion, exit the process.
[0146] The method for processing multiple data streams with equivalent indexes provided by the embodiment of the application comprises the following steps: determining a corresponding equivalent indexable expression chain according to an associated analysis expression, wherein the equivalent indexable expression chain comprises at least two equivalent indexable expressions, and the values of the equivalent indexable expressions are the same; constructing an equivalent index expression graph based on the equivalent indexable expression chain; traversing graph nodes in the equivalent index expression graph to determine an equivalent expression path of the equivalent index expression graph; taking the equivalent expression path as an index corresponding to a flow table; and analyzing and processing the data stream based on the index of the flow table. The method can perform equivalent index analysis on the associated analysis expression of the multiple data streams, so that the associated analysis of the multiple data streams can be quickly realized, and the efficiency of the analysis and processing of the multiple data streams is improved.
[0147] Corresponding to the method for processing multiple data streams with equivalent indexes, the application further provides a device for processing multiple data streams with equivalent indexes. Since the embodiments of the device are similar to the embodiments of the method, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The embodiments of the device for processing multiple data streams with equivalent indexes described below are only illustrative. Please refer to the accompanying drawings. Figure 17 As shown in the figure, it is a structure schematic diagram of the device for processing multiple data streams with equivalent indexes provided by the embodiment of the application.
[0148] The device for processing multiple data streams with equivalent indexes provided by the application comprises the following parts:
[0149] The equivalent indexable expression chain determination unit 1701 is configured to determine a corresponding equivalent indexable expression chain according to an associated analysis expression, wherein the associated analysis expression is used to represent the associated relationship between the parameters of the data stream stored in each flow table, and the equivalent indexable expression chain comprises at least two equivalent indexable expressions, and the values of the equivalent indexable expressions are the same.
[0150] The equivalent index expression graph construction unit 1702 is configured to construct an equivalent index expression graph based on the equivalent indexable expression chain, wherein the equivalent index expression graph comprises multiple graph nodes and edges between the graph nodes, the graph nodes comprise equivalent indexable expressions corresponding to the flow tables, and the edges comprise the associated relationship between the equivalent indexable expressions corresponding to the two connected graph nodes.
[0151] The equivalent expression path determination unit 1703 is configured to traverse the graph nodes in the equivalent index expression graph to determine an equivalent expression path of the equivalent index expression graph.
[0152] The data analysis and processing unit 1704 is configured to take the equivalent expression path as an index corresponding to a flow table, and analyze and process the data stream based on the index of the flow table.
[0153] Further, the equivalent index expression graph construction unit is specifically configured to: traverse the equivalent indexable expression chain to obtain an equivalent indexable expression; determine a first expression of a first type of the equivalent indexable expression and a graph node of a first type corresponding to the first expression as a first element in the equivalent index expression graph; the first element is used to indicate a storage location of information of a flow table corresponding to the first expression; determine a second expression of a second type of the equivalent indexable expression and a graph node of a second type corresponding to the second expression as a second element in the equivalent index expression graph; the second element is used to indicate a storage location of information of a flow table corresponding to the second expression; obtain the equivalent index expression graph according to the first element and the second element; wherein the graph node comprises a first identifier of a flow table and an equivalent indexable expression corresponding to the first identifier.
[0154] Further, the equivalent expression path determination unit is specifically configured to: traverse all graph nodes in the equivalent index expression graph to determine at least one path composed of a graph node and a corresponding edge; determine a path including all flow tables in the at least one path as the equivalent expression path.
[0155] Further, the equivalent indexable expression chain determination unit is specifically configured to: convert the stored data flows into a logical tree based on an association relationship between parameters of the stored data flows in each flow table; traverse nodes in the logical tree to determine a first node used for performing characteristic calculation; determine whether a first child node of the first node and left and right child nodes of the first child node constitute an indexable expression; the two flow table identifiers included in the indexable expression are different; in a case where it is determined that the first child node of the first node, the left child node and the right child node of the first child node constitute the indexable expression, add an expression corresponding to the left child node, an identifier of a flow table corresponding to the left child node, an expression corresponding to the right child node, and an identifier of a flow table corresponding to the right child node to an initial expression chain; the initial expression chain is a preset association structure expression used for storing expressions and identifiers; in a case where all nodes in the logical tree are traversed, determine the initial expression chain as the equivalent indexable expression chain.
[0156] Further, the determining whether the first node, the first child node of the first node, the left child node of the first child node and the right child node of the first child node form an indexable expression comprises: in a case where it is determined that the first child node is a child node representing an equal operation, determining whether the left child node and all child nodes related to the left child node are first single-table field expression nodes and whether the right child node and all child nodes related to the right child node are second single-table field expression nodes; in a case where it is determined that the left child node and all child nodes related to the left child node are the first single-table field expression nodes and the right child node and all child nodes related to the right child node are the second single-table field expression nodes, determining whether the first node, the first child node of the first node, the left child node of the first child node and the right child node of the first child node form an indexable expression based on an identifier of a flow table corresponding to the first single-table field expression node and an identifier of a flow table corresponding to the second single-table field expression node.
[0157] Further, the determining whether the first node, the first child node of the first node, the left child node of the first child node and the right child node of the first child node form an indexable expression based on the identifier of the flow table corresponding to the first single-table field expression node and the identifier of the flow table corresponding to the second single-table field expression node comprises: in a case where it is determined that the left child node and all child nodes related to the left child node are the first single-table field expression nodes, updating a first number of times that the first single-table field expression refers to a first table field based on the identifier of the flow table corresponding to the first single-table field expression node; in a case where it is determined that the right child node and all child nodes related to the right child node are the second single-table field expression nodes, updating a second number of times that the second single-table field expression refers to a second table field based on the identifier of the flow table corresponding to the second single-table field expression node; in a case where it is determined that the first number of times is greater than zero, the second number of times is greater than zero, and the identifier of the flow table corresponding to the first single-table field expression node is different from the identifier of the flow table corresponding to the second single-table field expression node, determining that the first node, the first child node of the first node, the left child node of the first child node and the right child node of the first child node form an indexable expression.
[0158] Further, the performing analysis and processing on the data flow based on the index of the flow table comprises: obtaining an index value corresponding to the data flow to be analyzed and an identifier of a flow table corresponding to the data flow to be analyzed; performing data flow correlation analysis based on the index value corresponding to the data flow to be analyzed and the index of the flow table corresponding to the identifier of the flow table to obtain a corresponding data set; and performing processing on the data flow to be analyzed based on a preset network security analysis and processing procedure and the data set.
[0159] Further, the preset network security analysis processing flow and the data set are used to process the to-be-analyzed data flow, specifically including: based on a preset deletion processing flow, deleting the to-be-deleted data flow associated with the index from the flow table corresponding to the flow table identifier in the index table slot, the to-be-deleted data flow including a timeout data flow; or, based on a preset insertion processing flow, adding the to-be-inserted data flow associated with the index to a preset hit table; or, based on a preset update processing flow, performing update processing on the to-be-updated data flow associated with the index; the data set includes the to-be-deleted data flow, the to-be-inserted data flow, or the to-be-updated data flow; the hit table stores data associated with successful analysis.
[0160] The multi-data flow processing device with equivalent index provided by the embodiment of the application determines the corresponding equivalent indexable expression chain through the associated analysis expression, the equivalent indexable expression chain includes at least two equivalent indexable expressions, and the value of each equivalent indexable expression is the same; the equivalent indexable expression graph is constructed based on the equivalent indexable expression chain, the equivalent expression path of the equivalent indexable expression graph is determined by traversing the graph node in the equivalent indexable expression graph, and the equivalent expression path is used as the index corresponding to the flow table; and the data flow is analyzed and processed based on the index of the flow table. The method can perform equivalent index analysis on the associated analysis expression of the multi-data flow, so as to quickly realize the associated analysis of the multi-data flow, thereby improving the efficiency of the multi-data flow analysis processing.
[0161] Corresponding to the multi-data flow processing method with equivalent index provided above, the application further provides an electronic device. Since the embodiment of the electronic device is similar to the embodiment of the above method, it is described relatively simply, and the related parts can be referred to the description of the above method embodiment. The electronic device described below is only illustrative. For example, Figure 18 As shown, it is a physical structure schematic diagram of an electronic device disclosed by the embodiment of the application. The electronic device can include a processor 1801, a memory 1802 and a communication bus 1803, wherein the processor 1801, the memory 1802 complete mutual communication through the communication bus 1803, and communicate with the outside through a communication interface 1804. The processor 1801 can call the logical instructions in the memory 1802 to execute the multi-data stream processing method with equivalent index, which includes: determining the corresponding equivalent indexable expression chain according to the correlation analysis expression; the correlation analysis expression is used to represent the correlation between the parameters of the data stream stored in each flow table; the equivalent indexable expression chain includes at least two equivalent indexable expressions, and the values of each equivalent indexable expression are the same; constructing an equivalent index expression graph based on the equivalent indexable expression chain; the equivalent index expression graph includes a plurality of graph nodes and edges between the graph nodes; the graph node includes the equivalent indexable expression corresponding to the flow table, and the edge includes the correlation between the equivalent indexable expressions corresponding to the two connected graph nodes; traversing the graph nodes in the equivalent index expression graph, determining the equivalent expression path of the equivalent index expression graph; taking the equivalent expression path as the index corresponding to the flow table, and analyzing and processing the data stream based on the index of the flow table.
[0162] In addition, the logical instructions in the memory 1802 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes a storage chip, a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.
[0163] In another aspect, the embodiments of the present application also provide a computer program product, which comprises a computer program stored on a processor readable storage medium, and the computer program comprises program instructions, when the program instructions are executed by a computer, the computer is capable of performing the multi-data stream processing method with equivalent index provided by the above-mentioned method embodiments. The method comprises: determining a corresponding equivalent indexable expression chain according to an associated analysis expression, the associated analysis expression is used to represent the association relationship between the parameters of the data stream stored in each flow table, the equivalent indexable expression chain comprises at least two equivalent indexable expressions, and the values of each equivalent indexable expression are the same; constructing an equivalent index expression graph based on the equivalent indexable expression chain, the equivalent index expression graph comprises a plurality of graph nodes and edges between the graph nodes, the graph nodes comprise the equivalent indexable expressions corresponding to the flow tables, and the edges comprise the association relationship between the equivalent indexable expressions corresponding to the two connected graph nodes; traversing the graph nodes in the equivalent index expression graph to determine an equivalent expression path of the equivalent index expression graph; and taking the equivalent expression path as the index corresponding to the flow table, and performing analysis processing on the data stream based on the index of the flow table.
[0164] In another aspect, the embodiments of the present application also provide a processor readable storage medium, and the processor readable storage medium stores a computer program, and the computer program is executed by a processor to implement the multi-data stream processing method with equivalent index provided by the above-mentioned embodiments. The method comprises: determining a corresponding equivalent indexable expression chain according to an associated analysis expression, the associated analysis expression is used to represent the association relationship between the parameters of the data stream stored in each flow table, the equivalent indexable expression chain comprises at least two equivalent indexable expressions, and the values of each equivalent indexable expression are the same; constructing an equivalent index expression graph based on the equivalent indexable expression chain, the equivalent index expression graph comprises a plurality of graph nodes and edges between the graph nodes, the graph nodes comprise the equivalent indexable expressions corresponding to the flow tables, and the edges comprise the association relationship between the equivalent indexable expressions corresponding to the two connected graph nodes; traversing the graph nodes in the equivalent index expression graph to determine an equivalent expression path of the equivalent index expression graph; and taking the equivalent expression path as the index corresponding to the flow table, and performing analysis processing on the data stream based on the index of the flow table.
[0165] The processor readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to a magnetic storage (such as a floppy disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical storage (such as a CD, a DVD, a BD, a HVD, etc.), and a semiconductor storage (such as a ROM, an EPROM, an EEPROM, a non-volatile memory (NAND FLASH), a solid state disk (SSD)), etc.
[0166] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.
[0167] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.
[0168] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.< / equalindexexpressionlist> < / equalindexexpressionlist> < / equalindexexpressionlist> As shown: it is buildIndexExpressionGraph. In the key data structure GraphNode, the variable tableID is defined as the flow table number corresponding to the current graph node, that is, the flow table identifier; the variable expression is defined as the graph node expression, which stores the index expression corresponding to the graph node, such as table1.sip; the variable linkNodes is defined as the linked node, which stores other GraphNode nodes linked by the current node. Specifically, (1) the input parameter key equivalent indexable expression linked list < / equalindexexpressionlist> < / tableid> < / tableid> < / tableid> < / keyedtableslot> < / data> < / index> < / keyedtableslot> < / keyvalue> < / data> < / data> < / streamclassifier> < / matchedtable> < / data> < / newdata> < / olddata> < / keyedtableslot> < / data> < / index> < / keyedtableslot> < / keyvalue> < / data> < / data> < / streamclassifier> < / matchedtable> < / data> < / data> < / matchedtable> < / joinfilter> < / joinfilter> < / joinrowslot> < / row> < / row> < / currenttable> < / row> < / currenttable> < / keyedrows> < / jointableslot> < / keyvalue> < / index> < / jointableslot> < / data> < / data> < / index> < / keyedtableslot> < / data> < / keyvalue> < / data> < / data> < / streamclassifier> < / data> < / data> < / data> < / newdata> < / olddata> < / data> < / data>
Claims
1. A method for processing multiple data streams with equality indexes, characterized in that, include: Based on the association analysis expression, determine the corresponding chain of equivalent indexable expressions; The association analysis expression is used to represent the association relationship between the parameters of the data streams stored in each flow table; the chain of equivalent indexable expressions includes at least two equivalent indexable expressions, and each of the equivalent indexable expressions has the same value; An equi-indexed expression graph is constructed based on the equi-indexable expression chain; the equi-indexed expression graph includes multiple graph nodes and edges between graph nodes; The graph node includes the equivalent indexable expression corresponding to the flow table, and the edge includes the association relationship between the equivalent indexable expressions corresponding to the two connected graph nodes; Traverse the graph nodes in the equality index expression graph to determine the equality expression path of the equality index expression graph; The path of the equivalent expression is used as the index of the flow table, and the data flow is analyzed and processed based on the index of the flow table. The step of determining the chain of equivalent indexable expressions corresponding to the association analysis expression specifically includes: Based on the relationships between the parameters of the data streams stored in each flow table, the stored data streams are transformed into a logic tree; Traverse the nodes in the logic tree to determine the first node used for representation calculation; Determine whether the first child node of the first node and the left and right child nodes of the first child node form an indexable expression; the flow table identifiers of the two flow tables included in the indexable expression are different; When it is determined that the first child node of the first node, the left child node of the first child node, and the right child node of the first child node constitute the indexable expression, the expression corresponding to the left child node, the identifier of the flow table corresponding to the left child node, the expression corresponding to the right child node, and the identifier of the flow table corresponding to the right child node are added to the initial expression chain; the initial expression chain is a preset associative structure expression used to store expressions and identifiers; After traversing all nodes in the logic tree, the initial expression chain is determined as the equivalent indexable expression chain; Determining whether the first child node of the first node and the left and right child nodes of the first child node form an indexable expression includes: If the first child node is determined to be a child node representing an equality operation, determine whether the left child node and all child nodes related to the left child node are first single-table field expression nodes, and determine whether the right child node and all child nodes related to the right child node are second single-table field expression nodes. If it is determined that the left child node and all child nodes related to the left child node are the first single-table field expression node, and the right child node and all child nodes related to the right child node are the second single-table field expression node, based on the identifier of the flow table corresponding to the first single-table field expression node and the identifier of the flow table corresponding to the second single-table field expression node, it is determined whether the first child node of the first node and the left and right child nodes of the first child node form an indexable expression.
2. The multi-data stream processing method with equality index according to claim 1, characterized in that, The construction of the equivalent indexed expression graph based on the equivalent indexable expression chain specifically includes: Traverse the chain of equivalent indexable expressions to obtain equivalent indexable expressions; The first expression of the first type of the equivalent indexable expression and the graph node of the first type corresponding to the first expression are determined as the first element of the equivalent indexable expression graph; the first element is used to indicate the storage location of the information of the flow table corresponding to the first expression; The second expression of the second type of the equivalent indexable expression and the graph node of the second type corresponding to the second expression are determined as the second element in the graph of the equivalent indexable expression; the second element is used to indicate the storage location of the information of the flow table corresponding to the second expression; Based on the first element and the second element, the equivalent index expression graph is obtained; The graph node includes a first identifier of the flow table and an equivalent indexable expression corresponding to the first identifier.
3. The multi-data stream processing method with equality index according to claim 1, characterized in that, The step of traversing the graph nodes in the equality index expression graph to determine the equality expression path of the equality index expression graph includes: Traverse all graph nodes in the graph of the equality index expression to determine at least one path consisting of the graph node and its corresponding edge; The path that includes all flow tables in the at least one path is determined as the equivalent expression path.
4. The multi-data stream processing method with equality index according to claim 1, characterized in that, The step of determining whether the first child node, the left child node, and the right child node of the first child node form an indexable expression based on the identifier of the flow table corresponding to the first single-table field expression node and the identifier of the flow table corresponding to the second single-table field expression node includes: If it is determined that the left child node and all child nodes related to the left child node are the first single-table field expression node, the first number of times the first single-table field expression references the first table field is updated based on the identifier of the flow table corresponding to the first single-table field expression node; If it is determined that the right child node and all child nodes related to the right child node are the second single-table field expression node, the second number of times the second single-table field expression references the second table field is updated based on the identifier of the flow table corresponding to the second single-table field expression node; If the first count is greater than zero, the second count is greater than zero, and the identifier of the flow table corresponding to the first single-table field expression node is different from the identifier of the flow table corresponding to the second single-table field expression node, then the first child node of the first node and the left and right child nodes of the first child node constitute an indexable expression.
5. The multi-data stream processing method with equality index according to claim 1, characterized in that, The network security analysis processing of the data stream to be analyzed based on the index of the flow table specifically includes: Obtain the flow table identifier corresponding to the data stream to be analyzed, and the index value corresponding to the data stream to be analyzed; Based on the index value corresponding to the data stream to be analyzed and the index of the flow table corresponding to the flow table identifier, data stream correlation analysis is performed to obtain the corresponding dataset; based on the preset network security analysis and processing flow and the dataset, the data stream to be analyzed is processed.
6. The multi-data stream processing method with equality index according to claim 5, characterized in that, The process of processing the data stream to be analyzed based on the preset network security analysis and processing flow and the dataset specifically includes: Based on a preset deletion process, the data stream to be deleted, which includes timed-out data streams, is deleted from the flow table corresponding to the flow table identifier in the index table slot; or, based on a preset insertion process, the data stream to be inserted, which is associated with the index, is added to a preset hit table; or, based on a preset update process, the data stream to be updated, which is associated with the index, is updated; the dataset contains the data stream to be deleted, the data stream to be inserted, or the data stream to be updated; the hit table stores data from successfully correlated data.
7. A multi-data stream processing apparatus with equality index, characterized in that, include: The equivalent indexable expression chain determination unit is used to determine the corresponding equivalent indexable expression chain based on the association analysis expression; The association analysis expression is used to represent the association relationship between the parameters of the data streams stored in each flow table; the chain of equivalent indexable expressions includes at least two equivalent indexable expressions, and each of the equivalent indexable expressions has the same value; An equi-indexed expression graph construction unit is used to construct an equi-indexed expression graph based on the chain of equi-indexable expressions; the equi-indexed expression graph includes multiple graph nodes and edges between graph nodes; the graph nodes include equi-indexable expressions corresponding to flow tables, and the edges include the association relationship between equi-indexable expressions corresponding to two connected graph nodes; An equivalence expression path determination unit is used to traverse the graph nodes in the equivalence index expression graph and determine the equivalence expression path of the equivalence index expression graph. The data analysis and processing unit is used to use the equivalent expression path as the index of the flow table, and to analyze and process the data flow based on the index of the flow table; The equivalent indexable expression chain determination unit is specifically used to: convert the stored data stream into a logic tree based on the association between the parameters of the data stream stored in each flow table; traverse the nodes in the logic tree to determine the first node used for representation calculation; Determine whether the first child node of the first node and the left and right child nodes of the first child node form an indexable expression; the flow table identifiers of the two flow tables included in the indexable expression are different; If the first child node of the first node, the left child node of the first child node, and the right child node of the first child node constitute the indexable expression, the expression corresponding to the left child node, the identifier of the flow table corresponding to the left child node, the expression corresponding to the right child node, and the identifier of the flow table corresponding to the right child node are added to the initial expression chain. The initial expression chain is a preset associative structure expression used to store expressions and identifiers; after traversing all nodes in the logic tree, the initial expression chain is determined as the equivalent indexable expression chain; Furthermore, determining whether the first child node of the first node and its left and right child nodes form an indexable expression includes: if the first child node is determined to be a child node representing an equality operation, determining whether the left child node and all child nodes related to the left child node are first single-table field expression nodes, and determining whether the right child node and all child nodes related to the right child node are second single-table field expression nodes; if the left child node and all child nodes related to the left child node are first single-table field expression nodes, and the right child node and all child nodes related to the right child node are second single-table field expression nodes, determining whether the first child node of the first node and its left and right child nodes form an indexable expression based on the identifier of the flow table corresponding to the first single-table field expression node and the identifier of the flow table corresponding to the second single-table field expression node.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the multi-data stream processing method with equal value index as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the multi-data stream processing method with equal value index as described in any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the multi-data stream processing method with equal value index as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Method for mathematical expression retrieval based on hierarchical indexing
CN104991905A
Multi-table connection optimization method in Gaia system
CN112256705A