Lock-free update method for conflict detection rules for network security access control policies
By employing a pre-allocated space table and a lock-free update method based on atomic operations in network security access control policies, the problems of low rule update efficiency and high concurrency impact are solved, achieving efficient and secure rule updates.
Patent Information
- Application Number
- CN202411878071.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-10-31
- Estimated Expiration
- 2044-12-19
AI Technical Summary
In existing network security access control strategies, rule update algorithms are inefficient and have a significant impact on concurrency, especially when using locking mechanisms, which can easily lead to thread starvation and performance degradation.
A lock-free update method using pre-allocated space tables and atomic operations is adopted. By pre-allocating space tables in memory and using atomic operations such as Fetch-And-Add and Compare-And-Swap, locks are avoided, enabling efficient rule updates.
It improves the efficiency of rule updates, reduces the impact on concurrency, ensures low latency for read threads and timely memory reclamation, and avoids the performance loss caused by locking mechanisms.
Smart Images
Figure CN119853973B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of network security technology, specifically relating to a lock-free update method for conflict detection rules for network security access control policies. Background Technology
[0002] Currently, most functions of network security protection systems involve executing rule-based security policies on access control components, such as firewalls, virus monitoring, trusted behavior monitoring, and host intrusion prevention. Regardless of where the access control component is implemented, what platform architecture it is based on, or what programming language is used, access control rules undoubtedly need to be set and updated. In a parallel computing environment, access control rules are undoubtedly a shared resource. The access control execution component needs to obtain and match rules, while the rule setting component needs to distribute and enable new rules while invalidating old ones.
[0003] Access control enforcement components typically exist within the context of a subject's access to an object and are executed concurrently. Therefore, if the access control rule update algorithm is not implemented well, it may seriously affect the overall performance of the system.
[0004] Typically, rule updates involve concurrency issues. For example, in a concurrent environment, the term "thread" is used to describe code objects that are scheduled in parallel. Based on this, assume the access control unit has the following characteristics: rules can be abstracted as pointers, denoted as Rn; the updating of access control rules, i.e., enabling new rules Ry and revoking old rules Rx, is implemented by an independent thread T0; the use of access control rules occurs in a concurrent environment of multiple threads, denoted as T1, T2…Tn; T0's rule update action does not occur frequently, and is a low-frequency event relative to rule usage.
[0005] Based on the above, the most basic algorithm for rule updating (Algorithm 1) uses a read-write lock mechanism, for example, the following logic is used to update T0:
[0006] AcquireForWite(Lock);
[0007] Free(R);
[0008] R = Ry;
[0009] ReleaseForWite(Lock);
[0010] The rules are obtained using the following logic in T1..n:
[0011] AcquireForRead(Lock);
[0012] Use(R);
[0013] ReleaseForRead(Lock);
[0014] Implementing Algorithm 1 has two problems: if the locks are not queued, and T1..n frequently acquires the read lock, then T0 may not be able to acquire the write lock and will be starved; the lock implementation mechanism may affect concurrency performance. For example, spin locks under Windows will increase the interrupt request level, thus affecting the scheduling of user threads.
[0015] Algorithm 2 is designed to improve upon Algorithm 1 by using a counter to control the release of rule pointers. Since incrementing / decrementing the counter and setting / getting the rule pointers are not atomic operations, a local lock is still necessary. T0 can use the following logic to implement the conversion from old to new rules, where ReferCount is a global variable of type int*, initialized to point to an integer containing 1:
[0016] Acquire(Lock);
[0017] (*ReferCount)--;
[0018] if ((*ReferCount) == 0) {
[0019] Free(R);
[0020] delete ReferCount;
[0021] }
[0022] R = Ry;
[0023] ReferCount = new int(1);
[0024] Release(Lock);
[0025] The following logic is used in T1..n to acquire, use, and release rules:
[0026] Acquire(Lock);
[0027] r = R;
[0028] c = ReferCount;
[0029] (*c)++;
[0030] Release(Lock);
[0031] Use(r);
[0032] Acquire(Lock);
[0033] if ((--(*c))==0){
[0034] Free(r);
[0035] delete c;
[0036] }
[0037] Release(Lock);
[0038] The advantage of Algorithm 2 is that there is no lock during Use(r), and the lock is local, resulting in a much smaller impact on performance. However, this algorithm still has the following problems: locks still exist, and acquiring locks still consumes time and may affect concurrency performance; it only considers the replacement of the old and new sets of rules. If the backlog effect of rule updates is to be accommodated, a more complex data structure, such as a queue, is required. However, accessing the queue position, counter, and rule pointer cannot be done atomically, so local locks are still unavoidable; the T0 thread still has the possibility of starving due to being unable to acquire the lock.
[0039] Access control rule updates are typically implemented using the RCU (Read-Copy-Update) synchronization mechanism. RCU is an improved read-write lock that allows read and write threads to run concurrently, thus preventing read threads from being blocked. The RCU mechanism still requires overall protection of the rule pointers during their use and a memory reclamation mechanism to handle memory management.
[0040] Since locks affect performance and can cause thread starvation, are there any lock-free methods? WRRM (Write-Rarely-Read-Many) is a lock-free algorithm based on Hazard pointers and Retired Lists, presented in the form of a mapping. This algorithm can avoid deadlocks and livelocks between read and write threads, exhibiting excellent performance. However, this lock-free algorithm still requires a memory reclamation phase, which is performed by the write thread or garbage collection thread. Furthermore, after each update, this algorithm often fails to promptly reclaim the most recently obsolete pointers, resulting in an indeterminate delay.
[0041] Another easily conceivable method is for T0 to set the rules and then leave them unmanaged. This involves simultaneously copying the rule to a linked list when it's set, then ignoring its release and using a dedicated thread for garbage collection after a certain period. However, this approach requires creating new threads, and the overhead of thread scheduling often outweighs the lock itself (after all, thread scheduling also requires escalating interrupt requests and using locks). Therefore, a dedicated garbage collection thread is not an option.
[0042] Consider another lock-free algorithm that aims to securely control and manipulate shared data using atomic operation instructions supported by the CPU. These atomic operation instructions are generally not considered to have locks at the algorithmic level, although they are typically implemented via the CPU's Lock# signal. CPU instructions that can atomically perform an operation have primitive representations including, but not limited to, the following: (1) Fetch-And-Add (FAA): atomically implements x = x + a, denoted as the function FAA(&x,a), whose return value is the changed value of x; (2) Compare-And-Swap (CAS): atomically implements if(x==a){x=y;y=old x;}, denoted as the function CAS(&x,a,y), whose return value is a Boolean variable indicating whether the operation was successful.
[0043] This lock-free algorithm extensively uses atomic operations to replace locks during its implementation. It is precisely because of these atomic operation capabilities, which are widely supported by most CPU architectures, that this invention can achieve an efficient, lock-free rule update algorithm. Summary of the Invention
[0044] (a) Technical problems to be solved
[0045] The technical problem to be solved by this invention is how to provide a lock-free update method for conflict detection rules for network security access control policies, so as to solve the problems of low efficiency and large impact on concurrency when using locks.
[0046] (II) Technical Solution
[0047] To address the aforementioned technical problems, this invention proposes a lock-free update method for conflict detection rules in network security access control policies. This method includes the following steps:
[0048] S1. A pre-allocation strategy is adopted to pre-allocate a space table in memory. The space table structure is defined as follows: struct CACHE_ALIGN CRulesManageUnitArray, where CACHE_ALIGN indicates that the structure memory allocation should be established on the integer boundary of the cache line; the structure CRulesManageUnitArray is composed of multiple basic units CRulesManageUnit, and each basic unit can store the related variables of one rule; the memory of the rules is counted and a reclamation bit Retire_Bit is established.
[0049] S2. Define a management unit structure CRulesAnchor and request a management unit RulesAnchor. The management unit is pinned in memory by a global variable. The role of the management unit CRulesAnchor is to manage CRulesManageUnit and CRulesManageUnitArray by setting their pointer variables Tail and Array as its own. Here, Tail is a pointer to the CRulesManageUnit type structure of the management unit, and Array is a space table defined in the queue.
[0050] S3. Allocate space tables and basic units for the new rules, including:
[0051] S31. First, check if the rule pointer is a null pointer, i.e., an empty rule. If it is not a null pointer, then it is a valid rule.
[0052] S32. Under the premise that the rule is a valid rule, determine whether the Array pointer of the current CRulesAnchor structure has been initialized, that is, whether the space table has been initialized. If it is a null pointer, it means that it does not point to the space table. At this time, memory units are allocated for the space table, and the memory structure is built on the integer boundary of the cache memory. If it is not a null pointer, the aligned memory will be reclaimed.
[0053] S33. After the spatial table is built, first, the CRulesManageUnit pointer p is set to point to the first basic unit of the spatial table. Then, it is determined whether the rule pointer contained in the first basic unit is a null pointer, that is, whether it points to a certain rule. If it is null, it is set to point to the new rule to be written, and the reference count is set to 1. If it is not null, the p pointer is set to point to the next basic unit, and the judgment is performed again. This process is repeated until the last basic unit. When the last basic unit is still not null, the NextArray pointer of the current spatial table is used to judge each basic unit of the next spatial table.
[0054] (III) Beneficial Effects
[0055] This invention proposes a lock-free update method for conflict detection rules in network security access control policies. The lock-free update method based on detection rules proposed in this invention mainly optimizes performance in the following aspects:
[0056] Dynamically allocated aligned memory allows the processor to access memory in only one operation instead of two, thereby improving efficiency.
[0057] Pre-allocate a space table to store data, reducing pop and push instructions, and after a thread is suspended, use the highest bit flag to determine the current cache line usage, without reclaiming memory, thus reducing memory allocation time.
[0058] In the CloseRules method, atomic operations DEC and CAS are used together to prevent the comparison value from changing due to interruption by other threads, which would cause the CAS operation to fail repeatedly. Attached Figure Description
[0059] Figure 1 This is a flowchart of the lock-free update method for the detection rules of the present invention.
[0060] Figure 2 The flowchart shows the access rule process for the lock-free update method of the detection rule. Detailed Implementation
[0061] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.
[0062] In view of this, and to address the aforementioned problems, this invention aims to implement a fast, secure, and stable lock-free update method for conflict detection rules in network security access control policies through a lock-free mechanism, thereby avoiding the problems of low efficiency and significant impact on concurrency that exist when using locks.
[0063] To achieve the above objectives, the present invention provides a lock-free update method for conflict detection rules in network security access control policies, comprising the following steps:
[0064] S1. A pre-allocation strategy is adopted, and a space table is pre-allocated in memory. The space table structure is defined as follows: struct CACHE_ALIGN CRulesManageUnitArray, where CACHE_ALIGN indicates that the structure memory allocation should be based on the integer boundary of the cache line. The structure CRulesManageUnitArray is composed of multiple basic units CRulesManageUnit, and each basic unit can store the related variables of a rule. The memory of the rule is counted and its highest bit is defined as the reclamation bit Retire_Bit. When Retire_Bit is 1, it means that this rule needs to be reclaimed. When Retire_Bit is 0, it means that this rule is valid.
[0065] The spatial table structure is defined as follows:
[0066] struct CACHE_ALIGN CRulesManageUnitArray{
[0067] A1 intptr_t ReferCount[CACHE_COUNT-1];
[0068] A2 intptr_t Padding;
[0069] A3 void*RulesPtr[CACHE_COUNT-1];
[0070] A4 CRulesManageUnitArray*NextArray;
[0071] };
[0072] The structure of a basic unit in a spatial table is defined as follows:
[0073] struct CRulesManageUnit{
[0074] A5 intptr_t ReferCount;
[0075] A6 intptr_t Padding[CACHE_COUNT-1];
[0076] A7 void*RulesPtr;
[0077] };
[0078] The CRulesManageUnitArray structure consists of multiple basic units, each of which can store variables related to one rule. However, if there are many rules, the structure may not be able to store all of them. Therefore, a pointer is set in the CRulesManageUnitArray structure to point to the next structure. This way, if the basic unit of the current structure has been used up, the basic unit of the next structure can be used through this pointer. The variables in CRulesManageUnitArray are the elements of the array.
[0079] A rule consists of two variables, which are called a basic unit. ReferCount is a reference count variable, representing how many threads are using this rule; RulesPtr is a rule pointer, which can be used to reference this rule.
[0080] The Retire_Bit is the highest bit of the "ReferCount" variable associated with a rule (ReferCount is stored in a cache line, which typically contains 64 bits, though this number varies across different architectures). Therefore, the value of Retire_Bit is contained within the ReferCount variable. In the C implementation of this lock-free update algorithm, the value of the highest bit, Retire_Bit, can be obtained using the following two lines of code:
[0081] int mask=1<<(sizeof(ReferCount)*8-1);
[0082] Return(ReferCount&mask)>>(sizeof(ReferCount)*8-1);
[0083] In order to clearly illustrate the role of the reclaim bit in rule updates and rule references, this article directly uses Retire_Bit to represent the highest bit of ReferCount.
[0084] Due to the special mechanism of caching, the smallest unit of operation is the cache line, rather than a single independent element.
[0085] The reference count variable ReferCount and the rule pointer Rulesptr are separated by a distance of CACHE_CO UNT-1, which ensures that ReferCount and Rulesptr are in different cache lines when the cache loads the CRulesManageUnit structure. This allows different threads that need their values to access them quickly, reducing the probability of a thread being slowed down due to a cache miss in a multi-threaded environment.
[0086] Among them, CACHE_ALIGN indicates that the structure memory allocation should be based on the integer boundary of the cache line, and the value set by CACHE_COUNT should cause RulesPtr to be allocated to the next page.
[0087] Considering that most cache lines are now 64 bits, CACHE_ALIG is predefined as alignas(64), and CHCHE_COUNT is predefined as 64 / sizeof(intptr_t).
[0088] Here, ReferCount is a reference count variable of type intptr_t. Defining it as intptr_t is for cross-platform portability. sizeof(intptr_t) represents the number of bits in the pointer. Therefore, CHCHE_COUNT represents the number of pointers that can be placed in a cache line.
[0089] Pre-allocating a cache block and defining it as a space table CRulesManageUnitArray can avoid the overhead of frequent memory allocation and release, and implement memory allocation in a general free list.
[0090] Unused memory blocks can be used directly. If all cache blocks are occupied by threads (a memory block refers to a region in memory, and a cache block refers to a memory block that has been allocated to the space table), then the system will request allocation again. After use, the memory block is not released directly, but is handed over to the free list for safekeeping.
[0091] In the spatial table structure, the pointer NextArray is a node pointer.
[0092] S2. Define a management unit structure CRulesAnchor and apply for a management unit RulesAnchor. The management unit is pinned in memory by a global variable. The role of the management unit CRulesAnchor is to manage (including searching, accessing, deleting, and modifying) CRulesManageUnit and CRulesManageUnitArray by setting their pointer variables Tail and Array as their own.
[0093] The management unit structure is as follows:
[0094] struct CRulesAnchor{
[0095] A8 CRulesManageUnit*Tail;
[0096] A9 CRulesManageUnitArray*Array;
[0097] };
[0098] Where Tail is a pointer to a CRulesManageUnit structure of the management unit.
[0099] Here, Array is a space table defined in the queue.
[0100] S3. Allocate space tables and basic units for the new rules, including:
[0101] S31. First, check if the rule pointer is a null pointer (i.e., an empty rule). If it is not a null pointer, it means that the rule is valid.
[0102] S32. Under the premise that the rule is a valid rule, determine whether the Array pointer of the current CRulesAnchor structure has been initialized, that is, whether the space table has been initialized. If it is a null pointer, it means that it does not point to the space table. At this time, memory units are allocated for the space table, and the memory structure is built on the integer boundary of the cache memory. If it is not a null pointer, the aligned memory will be reclaimed.
[0103] S33. After the spatial table is built, first, the CRulesManageUnit pointer p is set to point to the first basic unit of the spatial table. Then, it is determined whether the rule pointer contained in the first basic unit is a null pointer, that is, whether it points to a certain rule. If it is null, it is set to point to the new rule to be written, and the reference count is set to 1. If it is not null, the p pointer is set to point to the next basic unit, and the judgment is performed again. This process is repeated until the last basic unit. When the last basic unit is still not null, the NextArray pointer of the current spatial table is used to judge each basic unit of the next spatial table.
[0104] The new rule management unit structure is as follows:
[0105]
[0106]
[0107] The purpose of the B1 statement is to prevent incorrect references to null pointers.
[0108] Statements B4 to B6 define the pointer to the management unit structure, the space table, and the address to retrieve the queue, respectively.
[0109] Statements B10 to B16 determine if the current pointer array is null. If it is, the following operations are performed: Statement B12 dynamically allocates aligned memory, ensuring the processor only needs one operation instead of two, improving efficiency. Statement B13 fills the new_array memory block with a given value; here, the value is 0 to avoid the uncertainty introduced by random numbers. Statements B14 and B15 reclaim the aligned memory new_array if array is not null.
[0110] In statement B17, the starting address of the referenced variable is retrieved and stored in pointer p.
[0111] Statements B19 to B31 use the atomic CAS operation to determine if the rule pointed to by pointer p is a new rule. If it is, they further check if the reference variable pointed to by pointer p is being used by a thread. If the reference count is not zero, it means that a thread is already using it, and the highest bit Retire_Bit is set to 1 using the atomic FAA operation, indicating that its memory is being released. If the reference count is zero, it means that no thread is using it, and it can be directly incremented by 1. The setting of Retire_Bit avoids the premature circular use of pointers, thus avoiding the ABA problem in concurrency issues. Since this describes the rule update process, once a rule is updated, the old rule will definitely become invalid. That is, if a new thread wants to read this rule, it must read the new one, so the reference count of the old rule needs to be released. If the reference count is zero, it means that no thread is using this rule, so the reference count can be used for the new rule. To indicate that the new rule is valid, the reference count needs to be non-zero, so it needs to be incremented by 1.
[0112] In this context, statement B30 adds the offset amount to pointer p for each offset, and statement B32 redirects the pointer to the next array after use.
[0113] S4. Update the rule: Swap the pointer p of the new CRulesManageUnit structure and the address of &RulesAnchor.Tail through the atomic operation XCHG, so that &RulesAnchor.Tail points to the new rule;
[0114] The method for updating rules is defined as follows:
[0115] void UpdateRules(void*R)
[0116] {
[0117] C1 CRulesManageUnit*p;
[0118] C2 CRulesManageUnit*Unit=nullptr;
[0119] C3
[0120] C4 p = NewUnit(R);
[0121] C5 Unit=(CRulesManageUnit*)XCHG(&RulesAnchor.Tail,p);
[0122] C6 CloseRules(Unit);
[0123] }
[0124] In the method of updating rules, the key statement is C5, which uses the atomic operation XCHG to exchange the pointer p of the new CRulesManageUnit structure and the address of &RulesAnchor.Tail, so that &RulesAnchor.Tail points to the new rule. In statement C6, it is indicated that the memory will not be released after use, and it will be set to a null pointer.
[0125] S5. Closing and Reclaiming Rules: Determine if the basic unit pointer is null. If it is null, it means that it does not need to be closed and returns directly to improve efficiency. If it is not null, then after the reference count variable is decremented by one and is zero, compare the reference count variable with 0 through an atomic operation. If they are equal, call the DeleteUnit method to set the rule pointer of this basic unit to null and enable the RETIRE_BIT bit of the reference count of this basic unit.
[0126] The methods for closing and recycling rules are defined as follows:
[0127]
[0128] The D1 and D2 statements can return directly if the Unit pointer is null, thus improving execution efficiency.
[0129] The D6 and D10 statements are used to compare the reference count variable with 0 atomically when the count variable is zero. If they are equal, the DeleteUnit method is called to set the pointer to zero.
[0130] S6. Obtain the rule: Use a while loop to check if the current pointer is null. If it is null, return the null pointer directly; otherwise, return the latest rule unit.
[0131] The method for obtaining rules is defined as follows:
[0132]
[0133] The methods for obtaining rules mainly involve statements E4 and E8. A while loop is used to determine whether the current pointer is a null pointer. If it is a null pointer, it returns a null pointer directly. The judgment statement decrements the reference count by 1, indicating that if the highest bit is 1, the operation fails.
[0134] Example 1:
[0135] In the following description, since each rule update uses a different basic unit than the one before the rule update, the rule before the rule update is referred to as the new rule or the rule before the update. At the same time, since it is possible for the reference counts of the new rule, the new rule, and the second new rule to be non-zero at the same time, for the sake of simplicity, the following description only considers the process of the new rule replacing the new rule.
[0136] In the following description, cache line and cache page are the same concept and can be used interchangeably. A cache line or cache page represents the smallest unit that the cache loads from memory at a time. Each cache line or cache page contains multiple basic units, such as... Figure 1 As shown.
[0137] In the following description, relatively new rules require closing and recycling operations. Recycling refers to setting the rule pointer of the basic unit corresponding to the relatively new rule to null, so that this basic unit can be reused when new rules are made. Closing refers to releasing the memory space occupied by the rule object of the relatively new rule, preventing the memory space from growing larger and larger due to frequent rule updates. Closing and recycling operations are usually used together.
[0138] Figure 1 This is a flowchart of a lock-free update method for detection rules in some embodiments of the present invention. The lock-free update method includes the following steps for rule updating and rule access:
[0139] Taking a complete rule read / write operation as an example, the features of the present invention will be explained in detail.
[0140] Initially, there is no regular memory, no read / write threads, and a CRulesAnchor structure is defined (e.g., ...). Figure 1 ①), contains two pointers: an Array pointer pointing to a CRulesManageUnitArray structure (such as...). Figure 1 (②) The tail pointer points to the CRulesManageUnit structure (e.g., ... Figure 1 (③ in the middle)
[0141] At this point, there are no rules, and a thread is needed to perform rule update operations. Therefore, a basic unit of a spatial table (such as...) needs to be prepared for this rule. Figure 1 In section ②), this basic unit contains two pointers, one of which points to a regular object (such as...). Figure 1 (⑤) Another pointer points to the rule's reference count (i.e., how many threads are using this rule, such as...). Figure 1 (④ in the middle)
[0142] As shown in code B1-B16, when allocating space tables and basic units for new rules, the rule pointer is first checked to see if it is a null pointer (i.e., an empty rule). If it is not a null pointer, it means that the rule is valid. Assuming the rule is valid, as shown in lines B6 and B10, it is necessary to check if the Array pointer of the current CRulesAnchor structure has been initialized (i.e., if the space table has been initialized). If it is a null pointer, it means that it does not point to the space table. Therefore, as shown in lines B12 and B13, memory units need to be allocated for the space table (this step checks if the current pointer array is a null pointer; if it is, dynamically allocated aligned memory so that the processor only needs one operation instead of two, improving efficiency). The memory structure is built on the integer boundaries of the cache memory (i.e., on the integer boundaries of the cache line, and its allocated memory structure is as follows). Figure 1 As shown in the diagram, the space table structure is defined as `struct CACHE_ALIGN CRulesManageUnitArray` (as shown in code A1-A4, `CACHE_ALIGN` indicates that the structure memory allocation should be based on the integer boundary of the cache line, the distance between the reference count variable and the rule pointer, so that the cached loading structure is in different cache lines, allowing different threads that need its value to access it quickly. Here, the reference count variable and the rule pointer are in different cache lines). Subsequent operations load the entire cache line into the cache and perform pointer replacement and other operations on the basic units of the cache line, rather than loading only a basic unit. Therefore, the smallest unit of operation is the cache line, not a single independent element.
[0143] Because a single rule update uses only one basic unit of the spatial table (i.e., CRulesManageUnit, where the basic unit is as follows) Figure 1 As shown in the figure, the present invention adopts a pre-allocation strategy, which pre-allocates a space table in memory, and the space table contains multiple basic units according to the cache size.
[0144] After the spatial table is built, lines B4 and B17 first set the CRulesManageUnit pointer p to the first basic unit of the spatial table. Then, they check if the rule pointer contained in the first basic unit is null (i.e., whether it points to a certain rule). If it is null, it points to the new rule to be written in this invention, and the reference count is set to 1. If it is not null, the code on line B30 sets the p pointer to the next basic unit, and the check is performed again. This process is repeated until the last basic unit. When the last basic unit is still not null, the NextArray pointer of the current spatial table (i.e., the next spatial table) is used to check each basic unit of the next spatial table. In this way, this invention returns a pointer to the CRulesManageUnit structure (i.e., the pointer p of the basic unit allocated by this invention for the new rule).
[0145] With this pointer to the basic unit, the present invention can perform rule update operations. As shown in C4, the p pointer points to the basic unit allocated for the new rule R. Line C5 uses the XCHG atomic operation to make the Tail pointer of the global structure CRulesAnchor defined in the present invention point to p. Therefore, when other threads access a rule through RulesAnchor.Tail, they will access the newly created rule object. After the swap, the Unit pointer points to the basic unit corresponding to the next newer rule. This basic unit needs to be closed and recycled (the CloseRules function is executed, but it is not necessarily closed and recycled because other threads may be using it, i.e., the reference count is greater than 1).
[0146] Assuming that no other thread is referencing this new rule and the XCHG atomic operation has been completed, it is necessary to close and reclaim the basic unit corresponding to this new rule. As shown in the CloseRules function, it is first necessary to determine whether the pointer of this basic unit is a null pointer. If it is a null pointer, it means that it does not need to be closed, and it can be returned directly to improve efficiency. If it's not a null pointer, then in statement D6, if the reference count variable is zero after being decremented by D5, an atomic operation (comparing these two adjacent reference count variables to 0 is to prevent the operating system from scheduling other threads to access this rule and increment the reference count variable when D5 is zero) is used to compare the reference count variable with 0. If they are equal, the DeleteUnit method is called to set the rule pointer of this basic unit to null and activate the RETIRE_BIT bit of the reference count for this basic unit (the counting of rule memory and the establishment of the Retire_Bit indicates that this unit is no longer in use). This step uses an atomic operation to operate the memory pointer. Through the atomic operation, the reference count variable is compared with 0. If the reference count is 0, it is directly incremented by one, indicating that a new rule is being created; if it is not zero, it means that its RETIRE_BIT bit is active, and the atomic operation FAA resets the Retire_Bit and sets the reference count to 1. Among them, B27 is an atomic operation FAA that resets the Retire Bit and sets the reference count to 1; D6-D10 use atomic operations to perform memory pointer operations. Through atomic operations, the reference count variable is compared with 0. If the reference count is 0, it is directly incremented by one, which means that a new rule is created.
[0147] With the above method for closing and recycling rules, each time a thread accesses this rule, it is necessary to determine whether the rule needs to be closed and recycled after the access is completed. This is because the rule may have been updated at this time, and this thread is the last thread to access the rule before the update. Therefore, after the access is completed, the basic unit before the rule update needs to be closed and recycled.
[0148] Therefore, the method for a thread to obtain rules is shown in the OpenRules function. First, line E1 defines a pointer Unit to the CRulesManageUnit object to be returned. Then, line E5 executes the CloseRules function. Since Unit is a null pointer at this time, the CloseRules function in line E5 returns directly to line E6. Line E6 assigns the RulesAnchor.Tail pointer (i.e., the pointer to the last updated rule) to the Unit pointer. Line E8 checks whether the reference count of this basic unit is less than 0 after decrementing by one. If it is less than 0, it means that the operating system has scheduled another thread to update this rule between lines E6 and E8. The highest bit (reclaim bit) of this basic unit has been set to 1 (the highest bit being 1 represents a negative number, so it is less than 0). Therefore, the function body after do needs to be executed again. First, the CloseRules function in line E5 is executed to decrement the reference count of the old rule by one (after decrementing by one, if the reference count is 0, the memory is reclaimed in time). This operation ensures that the thread is not overwritten. Under the premise of termination, with the goal of minimizing the performance loss of the rule reading thread and achieving timely memory reclamation, line E6 assigns the latest RulesAnchor.Tail to the pointer Unit of the CRulesManageUnit object to be returned. Then, line E8 performs the reference count decrement operation to check if it is less than 0. If it is less than 0, it means that the rule has been updated again. The loop body is repeated (the while loop checks if the current pointer is null. If it is null, it returns null directly. The conditional statement decrements the reference count by 1, indicating that if the highest bit is 1, it fails) until the latest rule is obtained (at this time, the reference count is not less than 0 after incrementing, so the loop stops and line E10 directly returns this basic unit). Thus, the latest rule unit is obtained.
[0149] Example 2:
[0150] A lock-free update method for detection rules includes the following steps:
[0151] The memory structure is built on the integer boundary of the high-level cache memory, using a pre-allocation strategy. A space table is pre-allocated in memory, defined as: `struct CACHE_ALIGN CRulesManageUnitArray`, where `CACHE_ALIGN` indicates that the structure memory allocation should be based on the integer boundary of the cache line. Regular memory is counted, and a `Retire_Bit` is established. To ensure that threads are not terminated, and with the goal of minimizing performance loss for regular reading threads, timely memory reclamation is achieved. Atomic operations are used for memory pointer operations. Through atomic operations, the reference count variable is compared with 0. If the reference count is not zero, it means that a thread is already using it, and the `Retire_Bit` is set to 1 through the atomic operation `FAA`, indicating that the memory is being released. If they are equal, the corresponding method is called to set the pointer to zero.
[0152] Furthermore, the distance between the reference counting variable and the rule pointer allows the cached loading structure to be in different cache lines, enabling different threads that need its value to access it quickly, reducing the probability of a thread being slowed down due to a cache miss in a multi-threaded environment.
[0153] Furthermore, due to the special mechanism of caching, the smallest unit of operation in the lock-free update method based on detection rules is the cache line, rather than a single independent element.
[0154] Furthermore, it checks whether the current pointer array is a null pointer. If it is, it dynamically allocates aligned memory so that the processor only needs to access it once, instead of twice, thus improving efficiency.
[0155] Furthermore, an atomic operation is used to determine whether the rule pointed to by the pointer is a new rule. If so, it is further determined whether the reference variable pointed to by the pointer is being used by a thread. If the reference count is not zero, it means that a thread is already using it, and the highest bit Retire_Bit is set to 1 through an atomic operation to indicate that its memory is being released.
[0156] Furthermore, in the method definitions of the closing rule and the recycling rule, a while loop is used to determine whether the current pointer is a null pointer. If it is a null pointer, the null pointer is returned directly. The judgment statement decrements the reference count by 1, indicating that if the highest bit is 1, the operation fails.
[0157] This invention proposes a lock-free update method for access control rules, which can reclaim memory promptly while minimizing read thread latency, without requiring a dedicated write thread reclamation phase or a dedicated garbage collection thread. This invention avoids the overhead of frequent memory allocation and deallocation by pre-allocating a cache block in memory and defining it as a space table structure, thus allocating memory from a general available space table (free list). Unused memory blocks can be used directly. If a cache block is fully occupied by threads, it is then allocated from the system. After use, the memory block is not directly released but is instead stored in the free list. This invention aims to achieve a fast, safe, and stable access control rule update algorithm through a lock-free mechanism, avoiding the inefficiencies and significant concurrency impacts associated with using locks.
[0158] The lock-free update method based on detection rules proposed in this invention mainly optimizes performance in the following aspects:
[0159] Dynamically allocated aligned memory allows the processor to access memory in only one operation instead of two, thereby improving efficiency.
[0160] Pre-allocate a space table to store data, reducing pop and push instructions, and after a thread is suspended, use the highest bit flag to determine the current cache line usage, without reclaiming memory, thus reducing memory allocation time.
[0161] In the CloseRules method, atomic operations DEC and CAS are used together to prevent the comparison value from changing due to interruption by other threads, which would cause the CAS operation to fail repeatedly.
[0162] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A lock-free update method for conflict detection rules in network security access control policies, characterized in that, The method includes the following steps: S1. A pre-allocation strategy is adopted, and a space table is pre-allocated in memory. The space table structure is defined as follows: struct CACHE_ALIGN CRulesManageUnitArray, where CACHE_ALIGN indicates that the structure memory allocation should be based on the integer boundary of the cache line. The structure CRulesManageUnitArray is composed of multiple basic units CRulesManageUnit, and each basic unit can store the related variables of a rule. The memory of the rule is counted and its highest bit is defined as the reclamation bit Retire_Bit. When Retire_Bit is 1, it means that this rule needs to be reclaimed. When Retire_Bit is 0, it means that this rule is valid. S2. Define a management unit structure CRulesAnchor and request a management unit RulesAnchor. The management unit is pinned in memory by a global variable. The role of the management unit CRulesAnchor is to manage CRulesManageUnit and CRulesManageUnitArray by setting their pointer variables Tail and Array as its own. Here, Tail is a pointer to the CRulesManageUnit type structure of the management unit, and Array is a space table defined in the queue. S3. Allocate space tables and basic units for the new rules, including: S31. First, check if the rule pointer is a null pointer, i.e., an empty rule. If it is not a null pointer, then it is a valid rule. S32. Under the premise that the rule is a valid rule, determine whether the Array pointer of the current CRulesAnchor structure has been initialized, that is, whether the space table has been initialized. If it is a null pointer, it means that it does not point to the space table. At this time, memory units are allocated for the space table, and the memory structure is built on the integer boundary of the cache memory. If it is not a null pointer, the aligned memory will be reclaimed. S33. After the spatial table is built, first, the CRulesManageUnit pointer p is set to point to the first basic unit of the spatial table. Then, it is determined whether the rule pointer contained in the first basic unit is a null pointer, that is, whether it points to a certain rule. If it is null, it is set to point to the new rule to be written, and the reference count is set to 1. If it is not null, the p pointer is set to point to the next basic unit, and the judgment is performed again. This process is repeated until the last basic unit. When the last basic unit is still not null, the NextArray pointer of the current spatial table is used to judge each basic unit of the next spatial table.
2. The lock-free update method for conflict detection rules for network security access control policies as described in claim 1, characterized in that, A rule consists of two variables, called a basic unit. ReferCount is a reference count variable, representing how many threads are using this rule. RulesPtr is a rule pointer, which is used to reference this rule. The reference count variable ReferCount and the rule pointer RulesPtr are separated by a distance of CACHE_COUNT-1, so that ReferCount and RulesPtr are in different cache lines when the cache loading structure CRulesManageUnit is used.
3. The lock-free update method for conflict detection rules for network security access control policies as described in claim 2, characterized in that, The smallest unit of operation is the cache line, not a single individual element.
4. The lock-free update method for conflict detection rules for network security access control policies as described in claim 1, characterized in that, CACHE_ALIGN indicates that the structure memory allocation should be based on the integer boundary of the cache line, and the value set by CACHE_COUNT should make RulesPtr be allocated to the next page; when the cache line is 64 bits, CACHE_ALIGN is predefined as alignas(64), and CACHE_COUNT is predefined as 64 / sizeof(intptr_t); where ReferCount is a reference count variable of type intptr_t, and sizeof(intptr_t) represents the number of pointer bits. Therefore, CACHE_COUNT represents how many pointers can be placed in the cache line.
5. The lock-free update method for conflict detection rules for network security access control policies as described in claim 1, characterized in that, In step S33, the atomic operation CAS is used to determine whether the rule pointed to by pointer p is a new rule. If it is, the atomic operation FAA is used to determine whether the reference variable pointed to by pointer p is used by a thread. If the reference count is not zero, it means that a thread is already using it. The atomic operation FAA is used to set the highest bit Retire_Bit to 1, indicating that its memory is released. If the reference count is zero, it means that it is not being used by any thread, so increment it by 1.
6. The lock-free update method for conflict detection rules for network security access control policies as described in claim 5, characterized in that, The method also includes: a rule update process: by exchanging the pointer p of the new CRulesManageUnit structure and the address of &RulesAnchor.Tail through the atomic operation XCHG, so that &RulesAnchor.Tail points to the new rule.
7. The lock-free update method for conflict detection rules of network security access control policies as described in claim 6, characterized in that, The method also includes a closing rule and a recycling rule process: it determines whether the basic unit pointer is a null pointer. If it is a null pointer, it means that it does not need to be closed and returns directly to improve efficiency. If it is not a null pointer, it compares the reference count variable with 0 through an atomic operation after the counter variable is decremented by one. If they are equal, it calls the DeleteUnit method to set the rule pointer of this basic unit to null and makes the RETIRE_BIT bit of the reference count of this basic unit effective.
8. The lock-free update method for conflict detection rules for network security access control policies as described in claim 7, characterized in that, Assuming no other thread references this new rule and the XCHG atomic operation has been completed, the basic unit corresponding to this new rule needs to be closed and reclaimed. First, it is necessary to check if the pointer of this basic unit is null. If it is null, it means that it does not need to be closed, and it can be returned directly to improve efficiency. If it is not null, then after checking that the reference count variable is zero after being decremented by one, the reference count variable is compared with 0 through an atomic operation. If they are equal, the DeleteUnit method is called to set the rule pointer of this basic unit to null, and the RETIRE_BIT bit of the reference count of this basic unit is enabled. The rule memory is counted and a reclamation bit Retire_Bit is established, indicating that this unit is no longer in use.
9. The lock-free update method for conflict detection rules of network security access control policies as described in claim 5, characterized in that, The method also includes a rule acquisition process: a while loop is used to determine whether the current pointer is a null pointer. If it is a null pointer, the null pointer is returned directly; otherwise, the latest rule unit is returned.
10. The lock-free update method for conflict detection rules for network security access control policies as described in claim 1, characterized in that, The memory structure is built on the integer boundary of the high-level cache memory, using a pre-allocation strategy. A space table is pre-allocated in memory, defined as: `struct CACHE_ALIGN CRulesManageUnitArray`, where `CACHE_ALIGN` indicates that the structure memory allocation should be based on the integer boundary of the cache line. Regular memory is counted, and a `Retire_Bit` is established. To ensure that threads are not terminated, and with the goal of minimizing the performance loss of regular reading threads, timely memory reclamation is achieved. Atomic operations are used for memory pointer operations. Through atomic operations, the reference count variable is compared with 0; if the reference count is not zero, ... This indicates that a thread is already in use. The FAA will then use an atomic operation to set the Retire_Bit bit to 1. This indicates that the memory is being released, and if the values are equal, the corresponding method is called to set the pointer to zero.
Citation Information
Patent Citations
memory management system based on an MIC architecture processor
CN109582597A
File systems supporting data sharing
CN1291304A