A processing method, device and equipment for tracking a task list and a medium

Abstract

This invention provides a method, apparatus, device, and medium for processing tracking task tables. The method includes: acquiring data to be processed and a strategy for arranging connection tracking tasks; determining at least one connection tracking task according to the arrangement strategy; configuring at least one independent task queue corresponding to each of the at least one connection tracking task; configuring at least one independent task table corresponding to each of the at least one independent task queue; determining at least one data component to be processed according to the data to be processed; writing the at least one data component to be processed into the at least one independent task queue according to a preset matching rule; and processing the independent task table corresponding to the independent task queue. The independent task queue and task table design reduces waiting and synchronization overhead between processing units, enabling messages to be processed faster.

Description

Technical Field

[0001] This invention relates to the field of tracking task processing technology, and in particular to a method, apparatus, device, and medium for processing tracking task tables. Background Technology

[0002] In data communication networks, connection tracking (CT) is a crucial function responsible for monitoring and managing the connection status within the network, ensuring the correct forwarding of data packets and the stable operation of the network. With the continuous increase in network traffic and the dramatic rise in the number of connections, the size of the connection tracking table also expands, placing extremely high demands on the processing efficiency of connection tracking.

[0003] Traditional connection tracking implementations typically employ a centralized table structure to store and manage all connection states. This approach performs adequately when the number of connections is small, but as the number of connections increases, table access and update operations become more frequent, leading to severe lock contention issues. Specifically, when multiple processing threads or processes need to access or modify the connection tracking table simultaneously, they must compete to acquire the table's lock to ensure data consistency. However, the lock acquisition and release process not only adds overhead but can also cause processing threads or processes to wait and block, severely impacting the efficiency of connection tracking and overall network performance.

[0004] Furthermore, lock contention in the connection tracking table can lead to increased processing latency and decreased throughput. In highly competitive scenarios, processing threads or processes may need to wait for locks to be released for extended periods, resulting in increased packet processing latency and consequently impacting network service response speed and user experience. Simultaneously, lock contention also limits the system's ability to handle concurrent connections, making it difficult to improve throughput. Summary of the Invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide a method, apparatus, device and medium for processing a tracking task table that overcomes or at least partially solves the above problems.

[0006] To address the aforementioned problems, this invention discloses a method for processing a connection tracking task table, the method comprising:

[0007] A strategy for acquiring data to be processed and deploying connection tracking tasks; the connection tracking task is responsible for recording the connection status of data in the network.

[0008] Based on the deployment strategy of the connection tracking tasks, at least one connection tracking task is determined;

[0009] Configure at least one independent task queue corresponding to each of the at least one connection tracking tasks, based on the at least one connection tracking task;

[0010] Configure at least one independent task table corresponding to each independent task queue in the at least one independent task queue;

[0011] Based on the data to be processed, at least one data component to be processed is determined;

[0012] According to the preset matching rules, the at least one data component to be processed is written into the at least one independent task queue;

[0013] The independent task table corresponding to the independent task queue is processed according to the at least one data component to be processed in the at least one independent task queue.

[0014] Optionally, determining at least one data component to be processed based on the data to be processed includes:

[0015] Obtain the quintuple information of the data to be processed; the quintuple information includes the IP address of the sender of the data to be processed, the port number of the sender of the data to be processed, the IP address of the receiver of the data to be processed, the port number of the receiver of the data to be processed, and the transmission protocol number of the data packet to be processed.

[0016] Based on the quintuple information of the data to be processed and the preset quintuple information matching rules, at least one data component to be processed is determined.

[0017] Optionally, processing the independent task table corresponding to the independent task queue based on the at least one data component to be processed in the at least one independent task queue includes:

[0018] Obtain the quintuple information of the data component to be processed;

[0019] Update the independent task table corresponding to the independent task queue based on the quintuple information of the at least one data component to be processed in the at least one independent task queue.

[0020] Optionally, writing the at least one data component to be processed into the at least one independent task queue according to a preset matching rule includes:

[0021] The hash value of the data component to be processed is determined based on the quintuple information of the data component to be processed.

[0022] Based on the hash value and the preset matching rules, the at least one data component to be processed is written into the at least one independent task queue.

[0023] Optionally, determining at least one data component to be processed based on the quintuple information of the data to be processed and a preset quintuple information matching rule includes:

[0024] Obtain the first processing thread;

[0025] Based on the first processing thread, the five-tuple information of the data to be processed, and the preset five-tuple information matching rules, at least one data component to be processed is determined.

[0026] Optionally, writing the at least one data component to be processed into the at least one independent task queue according to the hash value and a preset matching rule includes:

[0027] Based on the first processing thread, the hash value, and the preset matching rules, the at least one data component to be processed is written into the at least one independent task queue.

[0028] Optionally, each of the at least one connection tracing tasks has a corresponding second processing thread, and updating the independent task table corresponding to the at least one independent task queue based on the 5-tuple information of the at least one data component to be processed in the at least one independent task queue includes:

[0029] Based on the quintuple information of the at least one data component to be processed in the at least one independent task queue and the second processing thread of the at least one independent task queue, the independent task table corresponding to the independent task queue is processed.

[0030] On the other hand, embodiments of the present invention disclose a processing apparatus for connecting and tracking task tables, the apparatus comprising:

[0031] The processing strategy acquisition module is used to acquire the data to be processed and the deployment strategy of the connection tracking task; the connection tracking task is a task responsible for recording the connection status of data in the network.

[0032] The tracking task deployment module is used to determine at least one connection tracking task according to the connection tracking task deployment strategy.

[0033] The task queue configuration module is used to configure at least one independent task queue corresponding to each of the at least one connection tracking tasks, based on the at least one connection tracking task.

[0034] The task table configuration module is used to configure at least one independent task table corresponding to each independent task queue in the at least one independent task queue, based on the at least one independent task queue.

[0035] A data component determination module is used to determine at least one data component to be processed based on the data to be processed.

[0036] The data component writing module is used to write the at least one data component to be processed into the at least one independent task queue according to a preset matching rule.

[0037] The task table update module is used to process the independent task table corresponding to the independent task queue based on the at least one unprocessed data component in the at least one independent task queue.

[0038] Optionally, the data component determination module includes:

[0039] The 5-tuple acquisition submodule is used to acquire the 5-tuple information of the data to be processed; the 5-tuple information includes the IP address of the sender of the data to be processed, the port number of the sender of the data to be processed, the IP address of the receiver of the data to be processed, the port number of the receiver of the data to be processed, and the transmission protocol number of the data packet to be processed.

[0040] The first data component processing submodule is used to determine at least one data component to be processed based on the quintuple information of the data to be processed and a preset quintuple information matching rule.

[0041] Optionally, the task table update module includes:

[0042] The component quintuple acquisition submodule is used to acquire the quintuple information of the data component to be processed;

[0043] The first task table processing submodule is used to update the independent task table corresponding to the independent task queue based on the quintuple information of the at least one data component to be processed in the at least one independent task queue.

[0044] Optionally, the data component writing module includes:

[0045] The hash value confirmation submodule is used to determine the hash value of the data component to be processed based on the quintuple information of the data component to be processed.

[0046] The first matching submodule is used to write the at least one data component to be processed into the at least one independent task queue according to the hash value and the preset matching rules.

[0047] Optionally, the first data component processing submodule includes:

[0048] The first thread confirmation unit is used to obtain the first processing thread;

[0049] The second data component processing unit is used to determine at least one data component to be processed based on the first processing thread, the quintuple information of the data to be processed, and a preset quintuple information matching rule.

[0050] Optionally, the first matching submodule includes:

[0051] The second matching unit is used to write the at least one data component to be processed into the at least one independent task queue according to the first processing thread, the hash value and the preset matching rule.

[0052] Optionally, each of the at least one connection tracing tasks has a corresponding second processing thread, and the first task table processing submodule includes:

[0053] The second task table processing unit is used to process the independent task table corresponding to the independent task queue based on the five-tuple information of the at least one data component to be processed in the at least one independent task queue and the second processing thread of the at least one independent task queue.

[0054] Accordingly, this invention discloses an electronic device, including: a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the various steps of the above-described embodiment of the connection tracking task table processing method.

[0055] Accordingly, embodiments of the present invention disclose a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the various steps of the above-described embodiment of the connection tracking task table processing method.

[0056] The embodiments of this invention offer the following advantages: By configuring an independent task queue and task table for each connection tracking task, the embodiments of this invention effectively avoid contention for access to the global connection tracking table by multiple processing units in traditional implementations, thereby completely eliminating lock contention issues. Since each connection tracking task has its own independent task queue and task table, the system can process multiple connection tracking tasks simultaneously without worrying about resource contention or waiting. This feature greatly enhances the system's concurrent processing capabilities, enabling it to more easily handle the demands of large-scale concurrent connections and packet processing. The independent task queue and task table design reduces waiting and synchronization overhead between processing units, allowing packets to be processed faster. This improvement reduces connection tracking processing latency, improves network service response speed, and thus provides users with a smoother and more efficient network experience. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the steps of an embodiment of a task tracking table processing method according to the present invention.

[0058] Figure 2 This is a flowchart illustrating the steps of another embodiment of the method for processing a tracking task table according to the present invention.

[0059] Figure 3 This is a schematic diagram of the PMD thread distributing CT tasks according to an embodiment of the processing method of the tracking task table of the present invention;

[0060] Figure 4 This is a timing diagram of the PMD thread without applying the processing method of the tracking task table of this invention;

[0061] Figure 5 This is a timing diagram of the PMD thread in an embodiment of a task tracking table processing method of the present invention;

[0062] Figure 6 This is a timing diagram of the PMD thread, which is an embodiment of another connection tracking task table processing method of the present invention.

[0063] Figure 7 This is a structural block diagram of an embodiment of a task tracking table processing device according to the present invention. Detailed Implementation

[0064] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] Connection tracking is a task that records, tracks, and manages the state of network connections during network communication.

[0066] When two endpoints begin communicating, the connection tracing task identifies and records the establishment of this new connection. This typically involves extracting key information from data packets, such as the source IP address, destination IP address, source port number, and destination port number, to uniquely identify the connection.

[0067] Connection State Maintenance: Throughout the connection's lifecycle, the connection tracing task continuously tracks and updates the connection's state. This includes recording packet transmission and reception, connection duration, and packet sequence numbers. This information is crucial for network management, troubleshooting, and security auditing. Performance Optimization and Load Balancing: By leveraging the information collected by the connection tracing task, network administrators can understand network traffic patterns and bottlenecks, enabling appropriate performance optimization and load balancing adjustments. The connection tracing task plays a vital role in network communication. It not only improves network security, reliability, and performance but also provides network administrators with rich network management information and data support.

[0068] OVS-DPDK stands for Open vSwitch with Data Plane Development Kit, which is a technology for using the Data Plane Development Kit (DPDK) in Open vSwitch (OVS).

[0069] Open vSwitch (OVS) is an open-source virtual switch that supports various virtualization technologies such as Xen, KVM, and VMware, and is widely used in scenarios such as cloud computing and network function virtualization (NFV).

[0070] The Data Plane Development Kit (DPDK) is a set of development platforms and interfaces developed by Intel for fast packet processing. It provides high-performance packet processing algorithms and user-space packet processing libraries, which can greatly improve data processing performance and throughput.

[0071] OVS-DPDK utilizes the high-performance packet processing library provided by DPDK to move packet processing from kernel mode to user mode. By bypassing the kernel protocol stack and using high-performance packet processing algorithms, OVS-DPDK achieves data plane acceleration.

[0072] One of the core concepts of this invention is to solve the lock contention problem when updating the CT table by using a method where each CT task has its own independent task table.

[0073] Reference Figure 1The diagram illustrates a step flowchart of an embodiment of a connection tracking task table processing method according to the present invention, which may specifically include the following steps:

[0074] Step 101: Obtain the data to be processed and the deployment strategy for the connection tracking task; the connection tracking task is responsible for recording the connection status of data in the network.

[0075] This step is the preparation phase, which requires acquiring two types of information. First, the packets. Second, the deployment strategy for connection tracing tasks. This strategy guides how to allocate and organize connection tracing tasks, and can be formulated based on various factors such as network topology, traffic patterns, system resources, and service needs.

[0076] Step 102: Determine at least one connection tracking task according to the deployment strategy of the connection tracking task;

[0077] Based on the deployment strategy obtained in step 101, determine which connection tracing tasks need to be performed. These tasks may include monitoring the status of specific network connections, recording the transmission path of data packets, detecting abnormal connections, etc., and can be dynamically adjusted according to business needs.

[0078] Step 103: Configure at least one independent task queue corresponding to each of the at least one connection tracking tasks, based on the at least one connection tracking task.

[0079] To process connection tracing tasks in parallel, the system configures a task queue for each task. These queues are used to store pending data related to the task, ensuring ordered data processing and isolation between tasks.

[0080] In one example, the queue could be a lock-free queue, a data structure that achieves thread safety without using locking mechanisms (such as mutexes or read-write locks). It ensures correctness and consistency in a multi-threaded environment through complex atomic operations (such as CAS operations). The design goal of lock-free queues is to provide high-performance enqueue and dequeue operations in high-concurrency scenarios, avoiding the performance overhead and potential deadlock problems associated with locking mechanisms. Implementations can be lock-free queues based on linked lists or arrays.

[0081] Step 104: Configure at least one independent task table corresponding to each independent task queue in the at least one independent task queue;

[0082] To correspond with the task queues, a separate task table needs to be configured for each task queue. These tables are used to store the status information, processing results, etc. of the connection tracking tasks, and are the core data structure for task execution.

[0083] Step 105: Determine at least one data component to be processed based on the data to be processed;

[0084] The data to be processed is decomposed into multiple data components. These components may be divided based on factors such as packet type, source, and destination, in order to be more efficiently allocated to different connection tracing tasks.

[0085] In one example, the data to be processed can be a collection of messages. In network communication, a message is the basic unit of data transmission, containing the information to be sent and possibly some control information such as source address, destination address, and protocol type. Messages can be any form of data, such as text, images, audio, video, or control commands, which are encapsulated in a specific format for transmission and processing in the network.

[0086] According to the preset segmentation method, the set of messages is divided into individual messages, and these messages are parsed to obtain their processing methods and corresponding matching information;

[0087] Step 106: Write the at least one data component to be processed into the at least one independent task queue according to the preset matching rules.

[0088] Based on preset matching rules, such as packet characteristics and task priority, the data components to be processed are allocated to the corresponding task queues. This step ensures that the data is accurately assigned to the connection tracing task responsible for processing it.

[0089] In one example, after obtaining the matching information for individual messages, these individual messages are allocated to the lockless queue of their corresponding CT tasks based on this matching information and preset matching rules.

[0090] Step 107: Process the independent task table corresponding to the independent task queue according to the at least one data component to be processed in the at least one independent task queue;

[0091] Based on the data components in the task queue, the corresponding task tables are processed. Processing actions include updating connection states, recording transmission information, and detecting anomalies. Through the independent task queue and task table design, the system can process multiple connection tracking tasks in parallel and efficiently, improving overall processing performance and response speed.

[0092] By configuring an independent task queue and task table for each connection tracking task, this embodiment of the invention effectively avoids contention for access to the global connection tracking table by multiple processing units in traditional implementations, thereby completely eliminating lock contention issues. Since each connection tracking task has its own independent task queue and task table, the system can process multiple connection tracking tasks simultaneously without worrying about resource contention or waiting. This feature greatly enhances the system's concurrent processing capabilities, enabling it to more easily handle the demands of large-scale concurrent connections and packet processing. The independent task queue and task table design reduces waiting and synchronization overhead between processing units, allowing packets to be processed faster. This improvement reduces connection tracking processing latency and improves network service response speed, thus providing users with a smoother and more efficient network experience.

[0093] Reference Figure 2 The diagram illustrates a step flowchart of another embodiment of the connection tracking task table processing method of the present invention, which may specifically include the following steps:

[0094] Step 201: Obtain the data to be processed and the deployment strategy for the connection tracking task; the connection tracking task is responsible for recording the connection status of data in the network.

[0095] This step is the preparation phase, which requires acquiring two types of information. First, the packets. Second, the deployment strategy for connection tracing tasks. This strategy guides how to allocate and organize connection tracing tasks, and can be formulated based on various factors such as network topology, traffic patterns, system resources, and service needs.

[0096] Step 202: Determine at least one connection tracking task according to the deployment strategy of the connection tracking task;

[0097] Based on the deployment strategy obtained in step 201, determine which connection tracing tasks need to be performed. These tasks may include monitoring the status of specific network connections, recording the transmission path of data packets, detecting abnormal connections, etc., and can be dynamically adjusted according to business needs.

[0098] Step 203: Configure at least one independent task queue corresponding to each of the at least one connection tracking tasks, based on the at least one connection tracking task.

[0099] To process connection tracing tasks in parallel, the system configures a task queue for each task. These queues are used to store pending data related to the task, ensuring ordered data processing and isolation between tasks.

[0100] In one example, the queue could be a lock-free queue, a data structure that achieves thread safety without using locking mechanisms (such as mutexes or read-write locks). It ensures correctness and consistency in a multi-threaded environment through complex atomic operations (such as CAS operations). The design goal of lock-free queues is to provide high-performance enqueue and dequeue operations in high-concurrency scenarios, avoiding the performance overhead and potential deadlock problems associated with locking mechanisms. Implementations can be lock-free queues based on linked lists or arrays.

[0101] Step 204: Configure at least one independent task table corresponding to each independent task queue in the at least one independent task queue;

[0102] To correspond with the task queues, a separate task table needs to be configured for each task queue. These tables are used to store the status information, processing results, etc. of the connection tracking tasks, and are the core data structure for task execution.

[0103] Step 205: Obtain the 5-tuple information of the data to be processed; the 5-tuple information includes the IP address of the sender of the data to be processed, the port number of the sender of the data to be processed, the IP address of the receiver of the data to be processed, the port number of the receiver of the data to be processed, and the transmission protocol number of the data packet to be processed.

[0104] A 5-tuple is a set of key attributes used to uniquely identify a network session or connection. It includes: Sender's IP address: the source IP address of the data packet; Sender's port number: the port used by the application or service at the source of the data packet; Receiver's IP address: the destination IP address of the data packet; Receiver's port number: the port used by the application or service at the destination of the data packet; Transport protocol number: a numerical identifier indicating the transport layer protocol (such as TCP, UDP, etc.) used by the data packet.

[0105] Step 206: Determine at least one data component to be processed based on the quintuple information of the data to be processed and the preset quintuple information matching rules.

[0106] During message processing, the system first obtains the five-tuple information of the message, and then checks this five-tuple information according to preset matching rules. Based on the matching results, the system determines the messages that need further processing and adds them to the tail of the corresponding lock-free queue so that subsequent CT concurrent threads can process them. In this way, the system can efficiently manage and process data packets or messages in the network based on the five-tuple information and matching rules.

[0107] In one embodiment, step 206 includes the following sub-steps:

[0108] Sub-step S11: Obtain the first processing thread;

[0109] In one instance, the first processing thread can be a PMD thread. The Poll Mode Driver (PMD) thread is a thread model in DPDK used to handle most of the work on the data path. It is responsible for receiving, classifying, and processing data packets. The PMD thread receives data packets by polling the input port and then classifies and processes them. It uses receive (RX) and transmit (TX) queues (often called rxqs and txqs) to manage the flow of data packets.

[0110] Sub-step S12: Based on the first processing thread, the quintuple information of the data to be processed, and the preset quintuple information matching rules, at least one data component to be processed is determined.

[0111] By combining the high-efficiency processing capability of the PMD thread with the accuracy of the five-tuple information matching rules, the network management system can process data packets quickly and accurately.

[0112] Step 207: Write the at least one data component to be processed into the at least one independent task queue according to the preset matching rules;

[0113] Based on preset matching rules, such as packet characteristics and task priority, the data components to be processed are allocated to the corresponding task queues. This step ensures that the data is accurately assigned to the connection tracing task responsible for processing it.

[0114] In one example, after obtaining the matching information for individual messages, these individual messages are allocated to the lockless queue of their corresponding CT tasks based on this matching information and preset matching rules.

[0115] In one embodiment, step 207 includes the following sub-steps:

[0116] Sub-step S21: Determine the hash value of the data component to be processed based on the quintuple information of the data component to be processed;

[0117] Sub-step S22: Write the at least one data component to be processed into the at least one independent task queue according to the hash value and the preset matching rule.

[0118] Preset matching rules can be based on the range of hash values, specific hash value patterns, or other conditions related to hash values. Specifically, they can be determined according to business needs.

[0119] By calculating hash values ​​and writing data components into independent task queues based on these hash values ​​and preset matching rules, the system is provided with a fast, flexible, and efficient data processing mechanism. This mechanism not only improves the speed and efficiency of data processing but also enhances the system's scalability, reliability, and flexibility.

[0120] In one embodiment, sub-step S22 further includes the following sub-step:

[0121] Sub-step S211: Write the at least one data component to be processed into the at least one independent task queue according to the first processing thread, the hash value and the preset matching rule.

[0122] The PMD thread determines which independent task queue to write at least one data component to be processed into, based on the calculated hash value and preset matching rules. This process is fast and efficient because the hash value provides a quick classification basis, while the preset matching rules ensure processing flexibility and accuracy.

[0123] Step 208: Process the independent task table corresponding to the independent task queue according to the at least one data component to be processed in the at least one independent task queue.

[0124] Based on the data components in the task queue, the corresponding task tables are processed. Processing actions include updating connection states, recording transmission information, and detecting anomalies. Through the independent task queue and task table design, the system can process multiple connection tracking tasks in parallel and efficiently, improving overall processing performance and response speed.

[0125] In one embodiment, step 208 includes the following sub-steps:

[0126] Sub-step S31: Obtain the quintuple information of the data component to be processed;

[0127] Before processing individual task tables, the system first needs to obtain the quintuple information of the data components to be processed.

[0128] The 5-tuple information typically includes the source IP address, source port number, destination IP address, destination port number, and protocol type, and is key information for identifying network data flows.

[0129] Obtaining the quintuple information is necessary to accurately update the individual task tables and track and process specific data streams in subsequent steps.

[0130] Sub-step S32: Update the independent task table corresponding to the independent task queue based on the quintuple information of the at least one data component to be processed in the at least one independent task queue.

[0131] After obtaining the 5-tuple information of the data component to be processed, the independent task table is updated based on this information. Updates may include recording the processing status of the data component, updating timestamps, adding relevant metadata or comments, etc. By updating the independent task table, the system can track and manage the status and progress of tasks in real time, providing decision support for subsequent processing.

[0132] By updating the independent task table based on the data components to be processed in the independent task queues, an accurate, scalable, and fine-grained task management mechanism is provided for the system. This mechanism not only improves the efficiency and accuracy of task processing but also enhances the reliability and maintainability of the system.

[0133] In one embodiment, each of the at least one connection tracking task has a corresponding second processing thread, and sub-step S32 includes the following sub-steps:

[0134] Sub-step S321: Based on the quintuple information of the at least one data component to be processed in the at least one independent task queue and the second processing thread of the at least one independent task queue, process the independent task table corresponding to the independent task queue:

[0135] The CT task thread is assigned to process the data in the current independent task queue. CT task threads are specifically designed to handle specific types of tasks or data streams, and each thread corresponds one-to-one with an independent task queue.

[0136] In one example, based on the acquired 5-tuple information and the identified CT task thread, the system updates the independent task table corresponding to the independent task queue. The update includes recording the processing status of the data component, the processing timestamp, the identifier of the CT task thread responsible for processing, and any other metadata or annotations related to task processing. The CT task thread processes the data component based on the information in the independent task table. During processing, the thread may update the task table to reflect changes in task progress or status. For example, when a task is completed, the thread updates the task status to "Completed" and records the completion time.

[0137] By configuring an independent task queue and task table for each connection tracking task, this embodiment of the invention effectively avoids contention for access to the global connection tracking table by multiple processing units in traditional implementations, thereby completely eliminating lock contention issues. Since each connection tracking task has its own independent task queue and task table, the system can process multiple connection tracking tasks simultaneously without worrying about resource contention or waiting. This feature greatly enhances the system's concurrent processing capabilities, enabling it to more easily handle the demands of large-scale concurrent connections and packet processing. The independent task queue and task table design reduces waiting and synchronization overhead between processing units, allowing packets to be processed faster. This improvement reduces connection tracking processing latency and improves network service response speed, thus providing users with a smoother and more efficient network experience.

[0138] Reference Figure 3 The diagram illustrates a PMD thread distributing CT tasks according to an embodiment of a tracking task table processing method of the present invention.

[0139] The left side shows multiple PMD modules, pmd 1, pmd 2, up to pmd K, which represent different physical data processing entry points. Data from each PMD module flows to a hash algorithm module. These circular nodes are labeled "hash" and are responsible for quickly calculating based on the characteristics of the data to determine which queue the data should be distributed to.

[0140] The diagram shows multiple queues, such as queue 1 and queue N, which act as intermediate buffers to temporarily store tasks distributed via a hash algorithm. These queues ensure that tasks wait in an orderly manner, avoiding chaos in data processing.

[0141] On the right is the core of CT task processing—the CT handler module, such as CT handler 1, CT handler 2, and CT handler N. Each CT handler is responsible for processing tasks in a specific queue, and each handler is equipped with a "CT table" to store or process task-related data.

[0142] The entire workflow is as follows: After receiving data, the PMD first uses a hash algorithm module to make a distribution decision, and then sends the data to the corresponding queue to wait. The data in the queue is eventually distributed to the corresponding CT handler for processing, while the CT table assists in this process to ensure the correct processing and storage of the data.

[0143] This architecture design achieves balanced task distribution through a hash algorithm, effectively improving the system's processing efficiency and throughput, avoiding overload of processing units, and ensuring efficient and stable data processing.

[0144] Reference Figure 4 The diagram illustrates the timing of the PMD thread without applying the processing method of the tracking task table of this invention.

[0145] First, when PMD1 begins processing the CT workflow, it sequentially enters the "Pre-CT Workflow" and "CT Workflow" phases. During the "CT Workflow" phase, PMD1 performs a series of add and delete operations on the CT table. These operations require a global CT table lock to ensure data consistency. Therefore, PMD1 holds this lock until its operations are complete.

[0146] However, while PMD1 holds the global lock, PMD2 and PMD3 must wait for PMD1 to release the lock so that they can also operate on the CT table. This waiting state is called a "blocked" state. Because PMD2 and PMD3 cannot continue to perform other tasks during the waiting period, this results in a waste of CPU processing power.

[0147] After PMD1 completes its operation and releases the global lock, PMD2 and PMD3 will sequentially enter the "CT process" stage to perform corresponding add / delete operations on the CT table. Then, they will sequentially enter the "post-CT process" stage for subsequent processing.

[0148] Reference Figure 5 The diagram illustrates the timing of the PMD thread in an embodiment of a task tracking table processing method according to the present invention.

[0149] First, the PMD modules (including PMD1, PMD2, and PMD3) are executed sequentially. Each module contains multiple cycles (such as PMD cycle i, PMD cycle i+1, etc.). These cycles represent different stages or tasks during the execution of the PMD module. After each PMD module completes its execution, the control flow enters the pre-CT process, which is a preparatory stage for entering the CT procedure.

[0150] In the original process, the CT process needed to wait for a global CT table lock upon entry. This meant that if the global lock was held by another process or thread, the CT process had to wait until the lock was released. This waiting resulted in a waste of CPU processing power, as the CPU could not perform other tasks during this time.

[0151] However, the execution method of the CT process has changed significantly in the improved process. Due to the application of the tracking task table processing method disclosed in this embodiment of the invention, the CT process no longer needs to preempt the global CT table lock upon entry. Therefore, the CT process can begin execution immediately after the PMD module finishes execution, without waiting for the global lock. This allows for more efficient utilization of CPU processing power, as the CPU can perform other tasks or operations during the period of waiting for the global lock to be released.

[0152] The arrow from PMD cycle i to the pre-CT process indicates that the pre-CT process begins after the PMD cycle ends; while the arrow from the pre-CT process to the CT process indicates that the CT process begins after the pre-CT process is completed, and this step is no longer restricted by the global lock after the improvement.

[0153] The diagram also illustrates the process relationships within the CT1 and CT2 modules. Each of the CT1 and CT2 modules contains multiple cycles (such as CT1 cycle i-1, CT1 cycle i, etc.; CT2 cycle i-1, etc.), which represent different stages or tasks in the execution process of the CT module.

[0154] Reference Figure 6 The diagram shows a timing diagram of the PMD thread in another embodiment of the connection tracking task table processing method of the present invention.

[0155] In this embodiment, the PMD thread will be responsible for the tasks of the CT thread, including CT task distribution, CT task processing, and CT task table updates. Figure 6 To put it another way:

[0156] The diagram mainly consists of three key parts: PMD1, PMD2, and PMD3, each representing a different processing module or thread. Each PMD module contains pre-CT processing and post-CT processing flows, while PMD2 and PMD3 also include additional CT processing flows.

[0157] The working mechanism of PMD threads:

[0158] Each PMD thread is responsible for processing its associated port receive queue. Within a processing cycle, the PMD thread first polls its associated receive queue once. After polling, the PMD thread checks if there are any tasks at the head of the CT task queue that need to be processed. If there are tasks, they are processed; otherwise, the process proceeds to the next PMD processing cycle.

[0159] CT concurrent threads can be merged with PMD threads to jointly process CT tasks. This means that in some cases, CT tasks can be handled directly by PMD threads without the need for separate CT concurrent threads.

[0160] The diagram uses dashed arrows to indicate an optional processing path. Specifically, when the PMD thread processes a CT task, if the calculated concurrent CT task number matches the CT task number currently held by the PMD thread, the step of inserting the task into the lock-free queue can be skipped, and subsequent processing can proceed directly. This optional path improves system efficiency and avoids unnecessary task insertions and queue operations.

[0161] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0162] Reference Figure 7 The diagram illustrates a structural block diagram of an embodiment of a processing device for connecting and tracking task tables according to the present invention, which may specifically include the following modules:

[0163] The processing strategy acquisition module 301 is used to acquire the data to be processed and the deployment strategy of the connection tracking task; the connection tracking task is a task responsible for recording the connection status of data in the network.

[0164] The tracking task placement module 302 is used to determine at least one connection tracking task according to the connection tracking task placement strategy.

[0165] The task queue configuration module 303 is used to configure at least one independent task queue corresponding to each of the at least one connection tracking tasks, based on the at least one connection tracking task.

[0166] The task table configuration module 304 is used to configure at least one independent task table corresponding to each independent task queue in the at least one independent task queue according to the at least one independent task queue;

[0167] The data component determination module 305 is used to determine at least one data component to be processed based on the data to be processed.

[0168] The data component writing module 306 is used to write the at least one data component to be processed into the at least one independent task queue according to a preset matching rule.

[0169] The task table update module 307 is used to process the independent task table corresponding to the independent task queue based on the at least one data component to be processed in the at least one independent task queue.

[0170] In one embodiment, the data component determination module includes:

[0171] The 5-tuple acquisition submodule is used to acquire the 5-tuple information of the data to be processed; the 5-tuple information includes the IP address of the sender of the data to be processed, the port number of the sender of the data to be processed, the IP address of the receiver of the data to be processed, the port number of the receiver of the data to be processed, and the transmission protocol number of the data packet to be processed.

[0172] The first data component processing submodule is used to determine at least one data component to be processed based on the quintuple information of the data to be processed and a preset quintuple information matching rule.

[0173] In one embodiment, the task table update module includes:

[0174] The component quintuple acquisition submodule is used to acquire the quintuple information of the data component to be processed;

[0175] The first task table processing submodule is used to update the independent task table corresponding to the independent task queue based on the quintuple information of the at least one data component to be processed in the at least one independent task queue.

[0176] In one embodiment, the data component writing module includes:

[0177] The hash value confirmation submodule is used to determine the hash value of the data component to be processed based on the quintuple information of the data component to be processed.

[0178] The first matching submodule is used to write the at least one data component to be processed into the at least one independent task queue according to the hash value and the preset matching rules.

[0179] In one embodiment, the first data component processing submodule includes:

[0180] The first thread confirmation unit is used to obtain the first processing thread;

[0181] The second data component processing unit is used to determine at least one data component to be processed based on the first processing thread, the quintuple information of the data to be processed, and a preset quintuple information matching rule.

[0182] In one embodiment, the first matching submodule includes:

[0183] The second matching unit is used to write the at least one data component to be processed into the at least one independent task queue according to the first processing thread, the hash value and the preset matching rule.

[0184] In one embodiment, each of the at least one connection tracing tasks has a corresponding second processing thread, and the first task table processing submodule includes:

[0185] The second task table processing unit is used to process the independent task table corresponding to the independent task queue based on the five-tuple information of the at least one data component to be processed in the at least one independent task queue and the second processing thread of the at least one independent task queue.

[0186] By configuring an independent task queue and task table for each connection tracking task, this embodiment of the invention effectively avoids contention for access to the global connection tracking table by multiple processing units in traditional implementations, thereby completely eliminating lock contention issues. Since each connection tracking task has its own independent task queue and task table, the system can process multiple connection tracking tasks simultaneously without worrying about resource contention or waiting. This feature greatly enhances the system's concurrent processing capabilities, enabling it to more easily handle the demands of large-scale concurrent connections and packet processing. The independent task queue and task table design reduces waiting and synchronization overhead between processing units, allowing packets to be processed faster. This improvement reduces connection tracking processing latency and improves network service response speed, thus providing users with a smoother and more efficient network experience.

[0187] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0188] This invention also provides an electronic device, comprising:

[0189] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described embodiment of the connection tracking task table processing method and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0190] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described connection tracking task table processing method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0191] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0192] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0193] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0194] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0196] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0197] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0198] The foregoing has provided a detailed description of a method, apparatus, device, and medium for processing a tracking task table provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for processing a connection tracking task table, characterized in that, The method includes: The strategy for acquiring data to be processed and the deployment of connection tracking tasks; the connection tracking task is a task responsible for recording the connection status of data in the network; the deployment strategy for the connection tracking task is used to guide how to allocate and organize connection tracking tasks; Based on the deployment strategy of the connection tracking tasks, at least one connection tracking task is determined; Configure at least one independent task queue corresponding to each of the at least one connection tracking tasks, based on the at least one connection tracking task; Configure at least one independent task table corresponding to each independent task queue in the at least one independent task queue; Based on the data to be processed, at least one data component to be processed is determined; According to the preset matching rules, the at least one data component to be processed is written into the at least one independent task queue; The independent task table corresponding to the independent task queue is processed according to the at least one data component to be processed in the at least one independent task queue.

2. The method according to claim 1, characterized in that, The step of determining at least one data component to be processed based on the data to be processed includes: Obtain the quintuple information of the data to be processed; the quintuple information includes the IP address of the sender of the data to be processed, the port number of the sender of the data to be processed, the IP address of the receiver of the data to be processed, the port number of the receiver of the data to be processed, and the transmission protocol number of the data packet to be processed. Based on the quintuple information of the data to be processed and the preset quintuple information matching rules, at least one data component to be processed is determined.

3. The method according to claim 2, characterized in that, The step of processing the independent task table corresponding to the independent task queue based on the at least one data component to be processed in the at least one independent task queue includes: Obtain the quintuple information of the data component to be processed; Update the independent task table corresponding to the independent task queue based on the quintuple information of the at least one data component to be processed in the at least one independent task queue.

4. The method according to claim 3, characterized in that, The step of writing the at least one data component to be processed into the at least one independent task queue according to a preset matching rule includes: The hash value of the data component to be processed is determined based on the quintuple information of the data component to be processed. Based on the hash value and the preset matching rules, the at least one data component to be processed is written into the at least one independent task queue.

5. The method according to claim 4, characterized in that, The step of determining at least one data component to be processed based on the quintuple information of the data to be processed and a preset quintuple information matching rule includes: Obtain the first processing thread; Based on the first processing thread, the quintuple information of the data to be processed, and the preset quintuple information matching rules, at least one data component to be processed is determined.

6. The method according to claim 5, characterized in that, The step of writing the at least one data component to be processed into the at least one independent task queue according to the hash value and the preset matching rule includes: Based on the first processing thread, the hash value, and the preset matching rules, the at least one data component to be processed is written into the at least one independent task queue.

7. The method according to claim 4, characterized in that, Each of the at least one connection tracing tasks has a corresponding second processing thread. Updating the independent task table corresponding to the at least one independent task queue based on the 5-tuple information of the at least one data component to be processed in the at least one independent task queue includes: Based on the quintuple information of the at least one data component to be processed in the at least one independent task queue and the second processing thread of the at least one independent task queue, the independent task table corresponding to the independent task queue is processed.

8. A processing apparatus for connecting to a tracking task list, characterized in that, The device includes: The processing strategy acquisition module is used to acquire the data to be processed and the deployment strategy of the connection tracking task; the connection tracking task is a task responsible for recording the connection status of data in the network; the deployment strategy of the connection tracking task is used to guide how to allocate and organize the connection tracking task. The tracking task deployment module is used to determine at least one connection tracking task according to the connection tracking task deployment strategy. The task queue configuration module is used to configure at least one independent task queue corresponding to each of the at least one connection tracking tasks, based on the at least one connection tracking task. The task table configuration module is used to configure at least one independent task table corresponding to each independent task queue in the at least one independent task queue, based on the at least one independent task queue. A data component determination module is used to determine at least one data component to be processed based on the data to be processed. The data component writing module is used to write the at least one data component to be processed into the at least one independent task queue according to a preset matching rule. The task table update module is used to process the independent task table corresponding to the independent task queue based on the at least one unprocessed data component in the at least one independent task queue.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the processing method for the connection tracking task table as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the processing method for the connection tracking task table as described in any one of claims 1 to 7.

Images