Data stream storage method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a data stream storage method and device, electronic equipment and storage medium. The method comprises: obtaining a first to-be-stored data stream and at least one storage module for storing the first to-be-stored data stream according to at least one type of storage rule; determining to store the first to-be-stored data stream into a first storage module according to a first storage rule; in the case that the number of hash bits of the first storage module reaches a maximum value, determining whether the number of wildcards of the hash bits in the data stream stored into the first storage module exceeds a preset threshold; in the case that the number of wildcards of the hash bits in the data stream stored into the first storage module exceeds the preset threshold, migrating the data stream corresponding to the hash bits whose wildcard number exceeds the preset threshold to a second storage module, which can make full use of the storage space of the storage module and solve the problem of low storage module filling rate.

Description

Technical Field

[0001] This invention relates to the field of storage technology, and more specifically to a method, apparatus, electronic device, and storage medium for storing data streams. Background Technology

[0002] With the rapid development of the internet age, network services and network security are receiving increasing attention from individuals, businesses, and nations. Individuals prioritize privacy, network latency, and speed; businesses focus on network traffic billing, customer network services, and network management; and for nations, network security is paramount. A prerequisite for this is sufficient accuracy and speed in filtering and screening network data streams. While software offers the most flexibility in providing various network services for network data streams, it falls far short of speed requirements. Therefore, hardware structures similar to Ternary Content Addressable Memory (TCAM) are needed to add Access Control List (ACL) rules to quickly determine specific types of network traffic. This, combined with server-side software platforms, achieves a win-win situation of flexible functionality and high-speed processing. However, in real-world scenarios, the unordered and dynamic addition of various rule types makes selecting the hash bits for a particular rule difficult, ultimately leading to an unsatisfactory fill rate. Currently, there is no effective solution to this problem. Summary of the Invention

[0003] In view of this, the main objective of the present invention is to provide a method, apparatus, electronic device and storage medium for storing data streams.

[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0005] This invention provides a method for storing a data stream, comprising:

[0006] Acquire a first data stream to be stored and at least one storage module that stores the first data stream to be stored according to at least one type of storage rule;

[0007] The first data stream to be stored is determined to be stored in the first storage module according to the first storage rule; the first storage rule is any one of the at least one type of storage rule; the first storage module is any one of the at least one storage module.

[0008] If the number of hash bits in the first storage module reaches the maximum value, determine whether the number of wildcard hash bits in the data stream stored in the first storage module exceeds a preset threshold.

[0009] If the number of wildcard hash bits in the data stream stored in the first storage module exceeds the preset threshold, the data stream corresponding to the hash bits whose number of wildcards exceeds the preset threshold is migrated to the second storage module; the second storage module is any storage module other than the first storage module among the at least one storage module.

[0010] In the above scheme, after migrating the data stream corresponding to the hash bits whose number of wildcards exceeds the preset threshold to the second storage module, the method further includes:

[0011] Determine whether the first storage module has storage space;

[0012] If the storage space exists in the first storage module, the second data stream to be stored is stored in the first storage module;

[0013] Determine whether the number of wildcard hash bits in the second data stream to be stored in the first storage module exceeds the preset threshold;

[0014] If the number of wildcard hash bits in the second data stream to be stored stored in the first storage module exceeds the preset threshold, the second data stream to be stored will be migrated to the second storage module.

[0015] In the above scheme, the method further includes:

[0016] If the number of wildcard hash bits in the second data stream to be stored stored in the first storage module does not exceed the preset threshold, the second data stream to be stored will continue to be stored in the first storage module.

[0017] In the above scheme, after migrating the second data stream to be stored to the second storage module, the method further includes:

[0018] Determine whether the second storage module has storage space;

[0019] If the storage space of the second storage module does not exist, determine the first hash bit of the second storage module;

[0020] Based on the first hash bit of the second storage module, the data stream stored in the second storage module is re-stored in the second storage module, and the third data stream to be stored is stored in the second storage module.

[0021] In the above scheme, the method further includes:

[0022] If the storage space is not available in the first storage module, the fourth data stream to be stored is stored in the second storage module based on the first hash bit of the second storage module.

[0023] In the above scheme, the first data stream to be stored includes a first data stream; determining to store the first data stream to be stored in the first storage module according to the first storage rule includes:

[0024] The first data stream is stored sequentially into the first storage module.

[0025] In the above scheme, the first data stream to be stored includes a second data stream; the step of determining to store the first data stream to be stored into the first storage module block according to the first storage rule includes:

[0026] Determine whether the first storage module has storage space;

[0027] If the storage space of the first storage module does not exist, determine the first hash bit of the first storage module;

[0028] Based on the first hash bit of the first storage module, the data stream stored in the first storage module is re-stored into the first storage module, and the second data stream is stored into the first storage module.

[0029] In the above scheme, the first data stream to be stored includes a third data stream; determining to store the first data stream to be stored in the first storage module according to the first storage rule includes:

[0030] If the storage space of the first storage module does not exist, determine the second hash bit of the first storage module;

[0031] Based on the first hash bit and the second hash bit of the first storage module, the data stream stored in the first storage module is re-stored into the first storage module, and the third data stream is stored into the first storage module.

[0032] This invention provides a data stream storage device, comprising:

[0033] An acquisition unit is used to acquire a first data stream to be stored and at least one storage module that stores the first data stream to be stored according to at least one type of storage rule;

[0034] The first storage unit is configured to determine whether to store the first data stream to be stored in the first storage module according to a first storage rule; the first storage rule is any one of the at least one type of storage rule; the first storage module is any one of the at least one storage module.

[0035] The first judgment unit is used to determine whether the number of wildcard hash bits in the data stream stored in the first storage module exceeds a preset threshold when the number of hash bits in the first storage module reaches the maximum value.

[0036] The first migration unit is configured to migrate the data stream corresponding to the hash bits whose number of wildcards exceeds the preset threshold to the second storage module when the number of wildcards in the data stream stored in the first storage module exceeds the preset threshold; the second storage module is any storage module other than the first storage module among the at least one storage module.

[0037] This invention provides a data stream storage device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in any of the above-mentioned embodiments.

[0038] This invention provides a storage medium storing executable instructions, which, when executed by a processor, implement the method described in any of the above embodiments.

[0039] This invention provides a method, apparatus, electronic device, and storage medium for storing data streams. The method includes: acquiring a first data stream to be stored and at least one storage module storing the first data stream according to at least one type of storage rule; determining to store the first data stream to be stored in a first storage module according to a first storage rule; the first storage rule being any one of the at least one type of storage rule; the first storage module being any one of the at least one storage modules; if the number of hash bits in the first storage module reaches a maximum value, determining whether the number of wildcard hash bits in the data stream stored in the first storage module exceeds a preset threshold; if the number of wildcard hash bits in the data stream stored in the first storage module exceeds the preset threshold, migrating the data stream corresponding to the hash bits whose wildcard count exceeds the preset threshold to a second storage module; the second storage module being any one of the at least one storage modules other than the first storage module. By migrating the data stream corresponding to the hash bits whose wildcard count exceeds the preset threshold to the second storage module when the number of hash bits in the first storage module reaches the maximum value, the storage space of the storage module can be fully utilized, thus solving the problem of low storage module fill rate. Attached Figure Description

[0040] Figure 1 This is a schematic diagram illustrating the working principle of ACL rule storage in software TCAM in the data stream storage method of this invention.

[0041] Figure 2 This is a schematic diagram illustrating the ACL rule storage principle of software TCAM in the data stream storage method of this invention.

[0042] Figure 3 This is a schematic diagram illustrating the implementation process of the data stream storage method according to an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram illustrating the working principle of ACL rule storage in the software TCAM of the data stream storage method in an embodiment of the present invention.

[0044] Figure 5 This is a schematic diagram illustrating a rule example of the data stream storage method implemented according to the ACL rules of the related technical software TCAM in an embodiment of the present invention.

[0045] Figure 6 This is a schematic diagram of an ACL rule implementation in the software TCAM, which is a data stream storage method according to an embodiment of the present invention.

[0046] Figure 7 This is a schematic diagram of the composition structure of the data stream storage device according to an embodiment of the present invention;

[0047] Figure 8 This is a schematic diagram of a hardware entity structure of a data stream storage device according to an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the specific technical solutions of the invention will be further described in detail below with reference to the accompanying drawings of the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0049] In related technologies, software TCAM uses two bits to implement software tri-state operations to simulate hardware tri-state operations, offering advantages such as low power consumption and relatively simple hardware implementation. ACL rules can consist of several 0s, 1s, and Xs, where X is a wildcard representing either 0 or 1. For example, assuming an ACL rule is "10X", the matching data streams would be "100" and "101". Due to the hardware characteristics of TCAM, the length of stored ACL rules is relatively fixed, generally customized according to user needs. The working principle of storing these rules is to use the distribution and quantity of 0s and 1s in the rule to determine the rule's location in the hardware storage module (i.e., selecting a specific bit as the hash bit of the storage module and calculating the hash value based on the hash bit).

[0050] Figure 1 This is a schematic diagram illustrating the working principle of ACL rule storage in software TCAM in the data stream storage method of this invention. Figure 2 This is a schematic diagram illustrating the ACL rule storage principle of software TCAM in the data stream storage method of this invention. For example... Figure 1 and Figure 2 As shown, assuming a storage module has 4 rows, each row can store 4 rules, and a total of 8 rules need to be stored in the storage module in sequence. Figure 2 (a) indicates that the first four rules are stored in order and placed directly in the row with the storage module address 0.

[0051] Figure 2(b) indicates that when storing the fifth rule, there is no space left in the first row to add a rule. Therefore, a hash bit needs to be selected. Each rule from rule one to rule five uses the hash value corresponding to this hash bit as its address, and the storage location of each rule is reset using the first and second rows. The 7th bit is selected as the first hash bit. The value of rule one in the 7th bit is X (i.e., the hash value). The wildcard means that it needs to be copied and distributed in the rows with storage module addresses 0 and 1 respectively. The hash values ​​of rule two and rule five are 1, so they are distributed in the row with storage module address 1. The hash values ​​of rule three and rule four are 0, so they are distributed in the row with storage module address 0.

[0052] Figure 2 (c) indicates that the sixth rule is stored. Since its hash value is 0, it can be directly stored in the row with storage module address 00. When storing the seventh rule, since its hash value is 1, it should be stored in the row with storage module address 01. When storing the eighth rule, since its hash value is X, it should be stored in rows 00 and 01. However, row 00 is already full, so a second hash bit needs to be selected to occupy rows 3 and 4 to redistribute the positions of rules 1 through 8. The second hash bit is chosen as bit 0. Therefore, the hash value of rule 1 is "XX", the hash value of rule 2 is "X1", the hash value of rule 3 is "10", the hash value of rule 4 is "00", the hash value of rule 5 is "11", the hash value of rule 6 is "00", the hash value of rule 7 is "01", and the hash value of rule 8 is "XX". The rules are then rearranged according to their hash values.

[0053] The storage process described in the aforementioned technologies allows rules to be dynamically populated in the storage module and indexed quickly, ensuring search speed. This approach is ideal for rules where wildcard and non-wildcard distribution areas are largely consistent. However, in real-world scenarios, various rule types are added dynamically and in an unordered manner, making it difficult to select the hash bits for a particular rule type, ultimately resulting in an unsatisfactory fill rate. Therefore, addressing the low fill rate problem is imperative to make full use of the TCAM space.

[0054] To address the shortcomings of the aforementioned related technologies, embodiments of the present invention provide a data stream storage method, apparatus, electronic device, and storage medium, attempting to solve the problem of low fill quantity due to disordered addition of various types of rules, thereby improving the fill rate of TCAM.

[0055] This invention proposes a data stream storage method. The function implemented by this method can be achieved by the processor in the data stream storage device calling program code. Of course, the program code can be stored in the computer storage medium. It can be seen that the computing device includes at least a processor and a storage medium.

[0056] Figure 3 This is a schematic diagram illustrating the implementation process of the data stream storage method according to an embodiment of the present invention, as follows: Figure 3 As shown, the method includes:

[0057] Step 301: Obtain a first data stream to be stored and at least one storage module that stores the first data stream to be stored according to at least one type of storage rule;

[0058] Step 302: Determine to store the first data stream to be stored in the first storage module according to the first storage rule; the first storage rule is any one of the at least one type of storage rule; the first storage module is any one of the at least one storage module;

[0059] Step 303: If the number of hash bits in the first storage module reaches the maximum value, determine whether the number of wildcard hash bits in the data stream stored in the first storage module exceeds a preset threshold.

[0060] Step 304: If the number of wildcard hash bits in the data stream stored in the first storage module exceeds the preset threshold, the data stream corresponding to the hash bits whose number of wildcards exceeds the preset threshold is migrated to the second storage module; the second storage module is any storage module other than the first storage module among the at least one storage module.

[0061] In step 301, the method for storing the data stream can be determined according to the actual situation and is not limited here. As an example, the method for storing the data stream can be a method for dynamically adjusting the distribution of ACL rules based on software TCAM.

[0062] The first data stream to be stored can be determined according to the actual situation, and is not limited here. As an example, the first data stream to be stored may be a data stream including the ACL rules.

[0063] The at least one type of storage rule can be determined based on actual circumstances and is not limited here. As an example, the at least one type of storage rule can be a storage rule corresponding to at least one type of data stream. The at least one type of data stream can be determined based on actual circumstances and is not limited here. As an example, the at least one type of data stream can be a data stream with at least one number of wildcards.

[0064] The at least one storage module can be determined based on actual conditions and is not limited here. As an example, the at least one storage module can be a storage module in a TCAM. The number of the at least one storage module can be determined based on actual conditions and is not limited here.

[0065] In step 302, the first storage rule can be determined according to the actual situation, and is not limited here. As an example, the first storage rule can be any of the storage rules corresponding to the at least one type of data stream.

[0066] The first storage module can be determined according to the actual situation, and is not limited here. As an example, the first storage module can be any of the storage modules in the at least one TCAM.

[0067] In step 303, the hash bits of the first storage module can be determined according to the actual situation, and are not limited here. As an example, any one or more bits of the data stream can be selected as the hash bits of the first storage module.

[0068] The maximum number of hash bits in the first storage module can be determined based on actual conditions and is not limited here. As an example, the maximum number of hash bits in the first storage module can be determined based on the number of storage rows in the first storage module.

[0069] The wildcard can be determined based on the actual situation and is not limited here. As an example, the wildcard can be the X character included in the ACL rule.

[0070] The hash bits in the data stream stored in the first storage module can be determined according to the actual situation and are not limited here. As an example, the hash bits in the data stream stored in the first storage module can be the hash values ​​corresponding to the hash bits of the data stream and the first storage module.

[0071] The preset threshold can be determined based on actual conditions and is not limited here. As an example, the preset threshold can be any value less than the maximum number of hash bits of the first storage module.

[0072] In step 304, migrating the data stream corresponding to the hash bits with the number of wildcards exceeding the preset threshold to the second storage module can be done by storing the data stream corresponding to the hash bits with the number of wildcards exceeding the preset threshold in the second storage module in sequence.

[0073] The second storage module can be determined according to actual conditions, and is not limited here. As an example, the second storage module can be any storage module other than the first storage module in the at least one TCAM.

[0074] This invention provides a data stream storage method. By migrating the data stream corresponding to hash bits whose wildcard count exceeds a preset threshold to a second storage module when the number of hash bits in the first storage module reaches its maximum value, the storage space of the storage module can be fully utilized, thus solving the problem of low storage module fill rate.

[0075] In an optional embodiment of the present invention, after migrating the data stream corresponding to the hash bits whose number of wildcards exceeds the preset threshold to the second storage module, the method further includes:

[0076] Determine whether the first storage module has storage space;

[0077] If the storage space exists in the first storage module, the second data stream to be stored is stored in the first storage module;

[0078] Determine whether the number of wildcard hash bits in the second data stream to be stored in the first storage module exceeds the preset threshold;

[0079] If the number of wildcard hash bits in the second data stream to be stored stored in the first storage module exceeds the preset threshold, the second data stream to be stored will be migrated to the second storage module.

[0080] In this embodiment, determining whether the first storage module has storage space can be done by determining whether the storage row corresponding to the second data stream to be stored in the first storage module has the storage space based on the hash bits of the first storage module.

[0081] When the storage space exists in the first storage module, storing the second data stream to be stored in the first storage module can be done by storing the second data stream to be stored in the storage row corresponding to the second data stream to be stored in the first storage module based on the hash bits of the first storage module when the storage space exists in the corresponding storage row of the first storage module.

[0082] The hash bits in the second data stream to be stored in the first storage module can be determined according to the actual situation, and are not limited here. As an example, the hash bits in the second data stream to be stored in the first storage module can be the hash values ​​corresponding to the hash bits of the second data stream to be stored and the hash bits of the first storage module.

[0083] The step of migrating the second data stream to be stored to the second storage module can be performed by storing the data stream corresponding to the hash bits whose number of wildcards exceeds the preset threshold into the second storage module in sequence, and then storing the second data stream to be stored into the second storage module in sequence.

[0084] In an optional embodiment of the present invention, the method further includes:

[0085] If the number of wildcard hash bits in the second data stream to be stored stored in the first storage module does not exceed the preset threshold, the second data stream to be stored will continue to be stored in the first storage module.

[0086] In this embodiment, the step of storing the second data stream to be stored in the first storage module can be to store the second data stream to be stored in the first storage module based on the hash bits of the first storage module.

[0087] In an optional embodiment of the present invention, after migrating the second data stream to be stored to the second storage module, the method further includes:

[0088] Determine whether the second storage module has storage space;

[0089] If the storage space of the second storage module does not exist, determine the first hash bit of the second storage module;

[0090] Based on the first hash bit of the second storage module, the data stream stored in the second storage module is re-stored in the second storage module, and the third data stream to be stored is stored in the second storage module.

[0091] In this embodiment, after migrating the second data stream to be stored to the second storage module, the method further includes: determining whether the storage space exists in the storage row corresponding to the third data stream to be stored in the first storage module based on the hash bits of the first storage module; if the storage space exists in the first storage module, storing the third data stream to be stored in the first storage module; if the number of wildcards in the hash bits of the third data stream to be stored stored in the first storage module exceeds the preset threshold, migrating the third data stream to be stored to the second storage module.

[0092] The determination of whether the second storage module has storage space can be made by determining whether the storage space exists in the storage row corresponding to the third data stream to be stored in the second storage module.

[0093] The first hash bit of the second storage module can be determined according to the actual situation, and is not limited here. As an example, any bit of the data stream can be selected as the first hash bit of the second storage module.

[0094] Based on the first hash bit of the second storage module, storing the data stream stored in the second storage module back into the second storage module, and storing the third data stream to be stored into the second storage module, can be performed as follows: Based on the first hash bit of the second storage module, determine the first hash bit in the data stream stored in the second storage module; based on the first hash bit in the data stream stored in the second storage module, store the data stream stored in the second storage module back into the second storage module; based on the first hash bit of the second storage module, determine the first hash bit in the third data stream to be stored; based on the first hash bit in the third data stream to be stored, store the third data stream to be stored into the second storage module.

[0095] The first hash bit in the data stream stored in the second storage module can be determined according to the actual situation, and is not limited here. As an example, the first hash bit in the data stream stored in the second storage module can be the hash value corresponding to the first hash bit of the data stream and the second storage module.

[0096] The first hash bit in the third data stream to be stored can be determined according to the actual situation, and is not limited here. As an example, the first hash bit in the third data stream to be stored can be the hash value corresponding to the first hash bit of the third data stream to be stored and the second storage module.

[0097] In an optional embodiment of the present invention, the method further includes:

[0098] If the storage space is not available in the first storage module, the fourth data stream to be stored is stored in the second storage module based on the first hash bit of the second storage module.

[0099] In this embodiment, the absence of the storage space in the first storage module can be determined by judging whether the storage row corresponding to the fourth data stream to be stored in the first storage module exists in the storage space based on the hash bits of the first storage module.

[0100] The step of storing the fourth data stream to be stored in the second storage module based on the first hash bit of the second storage module can be as follows: determining the first hash bit in the fourth data stream to be stored based on the first hash bit of the second storage module; and storing the fourth data stream to be stored in the second storage module based on the first hash bit in the fourth data stream to be stored.

[0101] The first hash bit in the fourth data stream to be stored can be determined according to the actual situation, and is not limited here. As an example, the first hash bit in the fourth data stream to be stored can be the hash value corresponding to the first hash bit of the fourth data stream to be stored and the second storage module.

[0102] In an optional embodiment of the present invention, the first data stream to be stored includes a first data stream; the step of determining to store the first data stream to be stored in the first storage module according to the first storage rule includes:

[0103] The first data stream is stored sequentially into the first storage module.

[0104] In this embodiment, storing the first data stream sequentially into the first storage module can be done by storing the first data stream sequentially into the first storage row of the first storage module.

[0105] The first storage row can be determined according to the actual situation, and is not limited here. As an example, the first storage row can be the storage row in the first storage module that first stores the data stream.

[0106] In an optional embodiment of the present invention, the first data stream to be stored includes a second data stream; the hash bits of the first storage module include the first hash bits of the first storage module; the step of determining to store the first data stream to be stored into the first storage module block according to the first storage rule includes:

[0107] Determine whether the first storage module has storage space;

[0108] If the storage space of the first storage module does not exist, determine the first hash bit of the first storage module;

[0109] Based on the first hash bit of the first storage module, the data stream stored in the first storage module is re-stored into the first storage module, and the second data stream is stored into the first storage module.

[0110] In this embodiment, determining whether the first storage module has storage space can be done by determining whether the second data stream has the storage space in the first storage row of the first storage module.

[0111] The first hash bit of the first storage module can be determined according to the actual situation, and is not limited here. As an example, any bit of the data stream can be selected as the first hash bit of the first storage module.

[0112] The step of storing the data stream stored in the first storage module back into the first storage module and storing the second data stream into the first storage module based on the first hash bit of the first storage module can be as follows: determining the first hash bit in the data stream stored in the first storage module based on the first hash bit of the first storage module; storing the data stream stored in the first storage module back into the first storage module based on the first hash bit in the data stream stored in the first storage module; determining the first hash bit in the second data stream based on the first hash bit of the first storage module; and storing the second data stream into the first storage module based on the first hash bit in the second data stream.

[0113] The first hash bit in the data stream stored in the first storage module can be determined according to the actual situation and is not limited here. As an example, the first hash bit in the data stream stored in the first storage module can be the hash value corresponding to the first hash bit of the data stream and the first hash bit of the first storage module.

[0114] The first hash bit in the second data stream can be determined according to the actual situation, and is not limited here. As an example, the first hash bit in the second data stream can be the hash value corresponding to the first hash bit of the second data stream and the first hash bit of the first storage module.

[0115] The step of re-storing the data stream stored in the first storage module to the first storage module and storing the second data stream to the first storage module can be performed by re-storing the data stream stored in the first storage module to the first storage row and the second storage row of the first storage module, and storing the second data stream to the first storage row and the second storage row of the first storage module.

[0116] The second storage line can be determined according to the actual situation, and is not limited here. As an example, the second storage line can be a storage line in the first storage module that stores the data stream after the first storage line.

[0117] In an optional embodiment of the present invention, the first data stream to be stored includes a third data stream; the hash bits of the first storage module include the second hash bits of the first storage module; the step of determining to store the first data stream to be stored in the first storage module according to the first storage rule includes:

[0118] If the storage space of the first storage module does not exist, determine the second hash bit of the first storage module;

[0119] Based on the first hash bit and the second hash bit of the first storage module, the data stream stored in the first storage module is re-stored into the first storage module, and the third data stream is stored into the first storage module.

[0120] In this embodiment, determining the second hash bit of the first storage module when the storage space of the first storage module does not exist can be done when the storage space of the first storage row and the second storage row of the first storage module does not exist.

[0121] The step of storing the data stream stored in the first storage module back into the first storage module and storing the third data stream into the first storage module based on the first hash bit and the second hash bit of the first storage module can be as follows: determining the first hash bit and the second hash bit of the data stream stored in the first storage module based on the first hash bit and the second hash bit of the first storage module; storing the data stream stored in the first storage module back into the first storage module based on the first hash bit and the second hash bit of the data stream stored in the first storage module; determining the first hash bit and the second hash bit of the third data stream based on the first hash bit and the second hash bit of the first storage module; and storing the third data stream into the first storage module based on the first hash bit and the second hash bit of the third data stream.

[0122] The first hash bit and the second hash bit of the data stream stored in the first storage module can be determined according to the actual situation, and are not limited here. As an example, the first hash bit and the second hash bit of the data stream stored in the first storage module can be the hash value corresponding to the first hash bit of the data stream and the hash value corresponding to the second hash bit of the data stream and the first storage module.

[0123] The first hash bit and the second hash bit in the third data stream can be determined according to the actual situation, and are not limited here. As an example, the first hash bit and the second hash bit in the third data stream can be the hash value corresponding to the first hash bit of the first storage module and the hash value corresponding to the second hash bit of the first storage module.

[0124] The step of storing the data stream stored in the first storage module back into the first storage module and storing the third data stream into the first storage module can be performed by storing the data stream stored in the first storage module back into the first storage row, second storage row, third storage row and fourth storage row of the first storage module, and storing the second data stream into the first storage row, second storage row, third storage row and fourth storage row of the first storage module.

[0125] The third and fourth storage lines can be determined based on actual circumstances and are not limited here. As an example, the third and fourth storage lines can be storage lines in the first storage module that store the data stream after the first and second storage lines.

[0126] To understand the embodiments of the present invention, the following description uses a method for dynamically adjusting the distribution of ACL rules based on software TCAM as an example.

[0127] Figure 4 This is a schematic diagram illustrating the working principle of ACL rule storage in the software TCAM of the data stream storage method according to an embodiment of the present invention. Figure 4 As shown, the method includes:

[0128] The first step is to initialize the storage modules. TCAM has k storage modules. Each storage module can store n rules per row, and each module has m rows. The space of each storage module is n×m. Each storage module can dynamically select a total of j hash bits, where j = [log₂m] hash bits. The initial number of hash bits for each storage module is i, where i = 0. A wildcard threshold p is set for each storage module (1 ≤ p ≤ j, p can be determined by the user based on actual conditions). The index w of the storage module is set, and w = 0 each time a new rule is stored.

[0129] The second step is to select a storage module and store the rules to be stored. Access the w-th storage module. Calculate the hash value of the rule in the w-th storage module based on its hash bits. If the storage row corresponding to the address of the indexed storage module has space, store the rule directly (prioritizing storage modules with fewer wildcards in the hash bits and available space at the corresponding location). Otherwise, proceed to the third step. If the third step fails, then w = w + 1. If the storage row corresponding to the address of the indexed storage module has no space, return failure and proceed to the second step.

[0130] The third step involves selecting a hash bit for a given storage module, redistributing the rules within that module, and storing the rules to be stored. If the first judgment satisfies i == j, proceed to the fourth step. If it's not the first time, return failure. This failure occurs either because the rule to be stored matched all storage modules or because an unqualified storage module was selected. The reason for failure is that a new hash bit cannot be generated, preventing the rule from being inserted. Otherwise, select the i-th hash bit. For each rule to be stored in this module, recalculate the hash value: hash value = (i-th hash bit << i) + hash value, where i = i + 1. Each rule in this module is redistributed according to its hash value index address, and finally, proceed to the second step.

[0131] Fourth step: If w+1 == k, the storage rule fails; otherwise, filter out the rules in the w-th storage module whose number of wildcards in the hash bits of that storage module exceeds p, and attempt to migrate the filtered rules to the w+1-th storage module. Proceed to the second step.

[0132] Figure 5 This is a schematic diagram illustrating a rule example of the data stream storage method implemented using the ACL rules of the related software TCAM, as described in an embodiment of the present invention. Figure 5 As shown, the process of storing rules 1 to f sequentially in the related technology includes:

[0133] Apply for the first storage module and directly store rules 1 to 4 in it. When storing rule 5, since it cannot be stored directly, select bit 0 as the first hash bit of the first storage module. Calculate the hash values ​​of rules 1 to 4 based on the first hash bit, redistribute rules 1 to 4 to the first two rows of the first storage module, and then store rule 5. Calculate the hash values ​​of rules 6 and 7 based on the first hash bit and store them directly in the first storage module. Rule 8 cannot be stored directly in the first storage module, so select bit 7 as the second hash bit of the first storage module. Calculate the hash values ​​of rules 1 to 7 based on the first and second hash bits, redistribute rules 1 to 7 to the first four rows of the first storage module, and then store rule 8. Rule 9 cannot be stored in the first storage module, and the number of hash bits in the first storage module has reached its maximum value, so a second storage module needs to be applied for.

[0134] In the second storage module, rules 9 through c are stored directly. Rule d cannot be directly stored in the second storage module, so the 8th bit is selected as the first hash bit of the second storage module. Rules 9 through c are hashed using the first hash bit, and then redistributed to rows one and two of the second storage module before storing rule d. Rule e can be directly stored in the first storage module. Rule f cannot be directly stored in either the first or second storage module, so the 9th bit is selected as the second hash bit of the second storage module. Rules 9 through d are hashed using the first and second hash bits, and then redistributed to rows one to four of the second storage module before storing rule f. Therefore, in this related technology, the first storage module has no remaining space, and the remaining space in the second module can only store a maximum of five rules.

[0135] Figure 6 This is a schematic diagram illustrating an example of an ACL rule implementation in the software TCAM, a data stream storage method according to an embodiment of the present invention. Figure 6 As shown, the process of storing ACL rules and their order in this embodiment includes:

[0136] The threshold p = 1 is set. The storage process for the first eight rules is the same as described above. When storing rule 9, a second storage module needs to be requested. Rules whose number of wildcards in the hash bits exceeds P in the first storage module are attempted to be stored in the second storage module. That is, rules 1 and 8 are migrated from the first storage module to the second storage module. The number of wildcards in the hash bits of rule 9 in the first storage module is no greater than p, and there is space to store rule 9, so rule 9 can be directly stored in the first storage module. Although rules a and b can be stored in the first storage module, the number of wildcards in their hash bits exceeds p. Considering that rules a and b can be directly stored in the second storage module, the storage of rules a and b in the first storage module is abandoned, and rules a and b are stored instead. The rules are stored in the second storage module. Although rule c can be stored in the first storage module, the number of wildcards in its hash bits exceeds p. Considering the second storage module, although its first storage row is full, its hash bits are less than j. Therefore, the 8th bit is chosen as the first hash bit of the full second storage module. The rules in the second storage module are redistributed, and rule c is saved. Rules d and e have wildcards less than p in the first storage module and corresponding storage locations, so rules d and e are stored in the first storage module. Rule f cannot be stored in the first storage module; it is saved to the second storage module based on the hash value obtained from the hash bits of the second storage module. It can be seen that in this embodiment, the first storage module has a maximum remaining space for storing six rules, and the second storage module has a maximum remaining space for storing ten rules. Comparing related technologies with this embodiment, it can be seen that the solution in this embodiment saves storage resources, allows data streams with similar patterns to be stored centrally, and improves the rule filling rate.

[0137] This embodiment addresses the low fill rate issue associated with dynamic addition of ACLs in multiple modes within the software TCAM. The main approach involves filtering rules in storage modules where the number of wildcards in the hash bits exceeds a user-defined p-value, and then migrating these rules to other storage modules. This dynamic adjustment method effectively improves the TCAM fill rate.

[0138] This invention provides a data stream storage device. Figure 7 This is a schematic diagram of the composition structure of the data stream storage device according to an embodiment of the present invention, as shown below. Figure 7 As shown, the device 700 includes:

[0139] The acquisition unit 701 is used to acquire a first data stream to be stored and at least one storage module that stores the first data stream to be stored according to at least one type of storage rule;

[0140] The first storage unit 702 is configured to determine that the first data stream to be stored is stored in the first storage module according to a first storage rule; the first storage rule is any one of the at least one type of storage rule; the first storage module is any one of the at least one storage module.

[0141] The first judgment unit 703 is used to determine whether the number of wildcard hash bits in the data stream stored in the first storage module exceeds a preset threshold when the number of hash bits in the first storage module reaches the maximum value.

[0142] The first migration unit 704 is configured to migrate the data stream corresponding to the hash bits whose number of wildcards exceeds the preset threshold to the second storage module when the number of wildcards in the data stream stored in the first storage module exceeds the preset threshold; the second storage module is any storage module other than the first storage module among the at least one storage module.

[0143] In other embodiments, after migrating the data stream corresponding to hash bits with a wildcard count exceeding the preset threshold to the second storage module, the device 700 further includes:

[0144] The second judgment unit is used to determine whether the first storage module has storage space;

[0145] The second storage unit is used to store the second data stream to be stored in the first storage module when the storage space exists in the first storage module;

[0146] The third judgment unit is used to determine whether the number of wildcards in the hash bits of the second data stream to be stored in the first storage module exceeds the preset threshold.

[0147] The second migration unit is used to migrate the second data stream to be stored to the second storage module when the number of wildcard hash bits in the second data stream to be stored stored in the first storage module exceeds the preset threshold.

[0148] In other embodiments, the second storage unit is further configured to maintain the storage of the second data stream to be stored in the first storage module as long as the number of wildcard hash bits in the second data stream to be stored stored in the first storage module does not exceed the preset threshold.

[0149] In other embodiments, after migrating the second data stream to be stored to the second storage module, the device 700 further includes:

[0150] The fourth determination unit is used to determine whether the second storage module has storage space;

[0151] The first determining unit is configured to determine the first hash bit of the second storage module when the storage space of the second storage module does not exist;

[0152] The third storage unit is used to restore the data stream stored in the second storage module back to the second storage module based on the first hash bit of the second storage module, and to store the third data stream to be stored in the second storage module.

[0153] In other embodiments, the device 700 further includes a third storage unit, used to store a fourth data stream to be stored in the second storage module based on the first hash bit of the second storage module when the first storage module does not have the storage space.

[0154] In other embodiments, the first data stream to be stored includes a first data stream; the first storage unit 702 is further configured to store the first data stream sequentially into the first storage module.

[0155] In other embodiments, the first data stream to be stored includes a second data stream; the first storage unit 702 is further configured to determine whether the first storage module has storage space; if the first storage module does not have storage space, determine the first hash bit of the first storage module; based on the first hash bit of the first storage module, restore the data stream stored in the first storage module to the first storage module, and store the second data stream to the first storage module.

[0156] In other embodiments, the first data stream to be stored includes a third data stream; the first storage unit 702 is further configured to determine the second hash bit of the first storage module when the first storage module does not have the storage space; based on the first hash bit and the second hash bit of the first storage module, to restore the data stream stored in the first storage module to the first storage module, and to store the third data stream to the first storage module.

[0157] The description of the above device embodiments is similar to that of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of the present invention, please refer to the description of the method embodiments of the present invention for understanding.

[0158] It should be noted that, in the embodiments of the present invention, if the above-described data stream storage method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a data stream storage device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of the present invention are not limited to any specific hardware and software combination.

[0159] Correspondingly, embodiments of the present invention provide a data stream storage device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements any step of the method described above.

[0160] Correspondingly, embodiments of the present invention provide a storage medium storing executable instructions, which, when executed by a processor, implement any step of the method described above.

[0161] It should be noted that the descriptions of the storage medium and device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of the present invention, please refer to the descriptions of the method embodiments of the present invention for understanding.

[0162] It should be noted that, Figure 8 This is a schematic diagram of a hardware entity structure of a data stream storage device according to an embodiment of the present invention, such as... Figure 8 As shown, the hardware entity of the data stream storage device 800 includes a processor 801 and a memory 803. Optionally, the data stream storage device 800 may also include a communication interface 802.

[0163] It is understood that memory 803 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 803 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0164] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 801. Processor 801 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 801 or by instructions in software form. The processor 801 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 801 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present invention can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 803. Processor 801 reads the information in memory 803 and completes the steps of the aforementioned method in conjunction with its hardware.

[0165] In an exemplary embodiment, the data stream storage device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0166] In the several embodiments provided by this invention, it should be understood that the disclosed methods and apparatus can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another observation, or some features may be ignored or not executed. In addition, the communication connections between the various components shown or discussed may be through some interfaces, indirect coupling or communication connections between devices or units, and may be electrical, mechanical, or other forms.

[0167] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0168] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0169] Alternatively, if the integrated units described above in the embodiments of the present invention are implemented as software functional units and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a data stream storage device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0170] The data stream storage method, apparatus, and computer storage medium described in this invention are only examples of the embodiments of this invention, but are not limited thereto. Any data stream storage method, apparatus, and computer storage medium involved are within the protection scope of this invention.

[0171] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of the invention, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the invention. The sequence numbers of the above-described embodiments of the invention are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0172] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0173] The above description is merely an embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for storing a data stream, characterized in that, include: Acquire a first data stream to be stored and at least one storage module that stores the first data stream to be stored according to at least one type of storage rule; The first data stream to be stored is determined to be stored in the first storage module according to the first storage rule; the first storage rule is any one of the at least one type of storage rule; the first storage module is any one of the at least one storage module. If the number of hash bits in the first storage module reaches the maximum value, determine whether the number of wildcard hash bits in the data stream stored in the first storage module exceeds a preset threshold. If the number of wildcard hash bits in the data stream stored in the first storage module exceeds the preset threshold, the data stream corresponding to the hash bits whose number of wildcards exceeds the preset threshold will be migrated to the second storage module. The second storage module is any storage module other than the first storage module among the at least one storage module; After migrating the data stream corresponding to the hash bits whose wildcard count exceeds the preset threshold to the second storage module, the method further includes: Determine whether the first storage module has storage space; If the storage space exists in the first storage module, the second data stream to be stored is stored in the first storage module; Determine whether the number of wildcard hash bits in the second data stream to be stored in the first storage module exceeds the preset threshold; If the number of wildcard hash bits in the second data stream to be stored stored in the first storage module exceeds the preset threshold, the second data stream to be stored will be migrated to the second storage module. After migrating the second data stream to be stored to the second storage module, the method further includes: Determine whether the second storage module has storage space; If the storage space of the second storage module does not exist, determine the first hash bit of the second storage module; Based on the first hash bit of the second storage module, the data stream stored in the second storage module is re-stored in the second storage module, and the third data stream to be stored is stored in the second storage module; If the storage space is not available in the first storage module, the fourth data stream to be stored is stored in the second storage module based on the first hash bit of the second storage module.

2. The method according to claim 1, characterized in that, The method further includes: If the number of wildcard hash bits in the second data stream to be stored stored in the first storage module does not exceed the preset threshold, the second data stream to be stored will continue to be stored in the first storage module.

3. The method according to claim 1, characterized in that, The first data stream to be stored includes a first data stream; determining to store the first data stream to be stored in the first storage module according to the first storage rule includes: The first data stream is stored sequentially into the first storage module.

4. The method according to claim 1, characterized in that, The first data stream to be stored includes a second data stream; determining to store the first data stream to be stored into the first storage module block according to the first storage rule includes: Determine whether the first storage module has storage space; If the storage space of the first storage module does not exist, determine the first hash bit of the first storage module; Based on the first hash bit of the first storage module, the data stream stored in the first storage module is re-stored into the first storage module, and the second data stream is stored into the first storage module.

5. The method according to claim 4, characterized in that, The first data stream to be stored includes a third data stream; determining to store the first data stream to be stored in the first storage module according to the first storage rule includes: If the storage space of the first storage module does not exist, determine the second hash bit of the first storage module; Based on the first hash bit and the second hash bit of the first storage module, the data stream stored in the first storage module is re-stored into the first storage module, and the third data stream is stored into the first storage module.

6. A data stream storage device, characterized in that, The storage device is configured to perform the method according to any one of claims 1 to 5, the storage device comprising: An acquisition unit is used to acquire a first data stream to be stored and at least one storage module that stores the first data stream to be stored according to at least one type of storage rule; The first storage unit is configured to determine whether to store the first data stream to be stored in the first storage module according to a first storage rule; the first storage rule is any one of the at least one type of storage rule; the first storage module is any one of the at least one storage module. The first judgment unit is used to determine whether the number of wildcard hash bits in the data stream stored in the first storage module exceeds a preset threshold when the number of hash bits in the first storage module reaches the maximum value. A first migration unit is configured to migrate the data stream corresponding to hash bits whose number of wildcards exceeds a preset threshold to a second storage module when the number of wildcards in the data stream stored in the first storage module exceeds a preset threshold; the second storage module is any storage module other than the first storage module among the at least one storage module. The device further includes: a second determining unit, configured to determine whether the first storage module has storage space; a second storage unit, configured to store the second data stream to be stored in the first storage module when the first storage module has storage space; a third determining unit, configured to determine whether the number of wildcard hash bits in the second data stream to be stored stored in the first storage module exceeds the preset threshold; and a second migration unit, configured to migrate the second data stream to be stored to the second storage module when the number of wildcard hash bits in the second data stream to be stored stored in the first storage module exceeds the preset threshold. The device further includes: a fourth determining unit, configured to determine whether the second storage module has storage space; a first determining unit, configured to determine the first hash bit of the second storage module when the second storage module does not have storage space; and a third storage unit, configured to re-store the data stream stored in the second storage module into the second storage module based on the first hash bit of the second storage module, and to store the third data stream to be stored into the second storage module. The third storage unit is used to store the fourth data stream to be stored in the second storage module based on the first hash bit of the second storage module when the first storage module does not have the storage space.

7. A data stream storage device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores executable instructions that, when executed by a processor, implement the method described in any one of claims 1 to 5.

Images