Data processing method and device, equipment and storage medium

Abstract

Embodiments of the present application provide a data processing method, device and equipment and a storage medium. The method comprises: in response to a service request, extracting features of a data packet in the service request to obtain tuple data; the tuple data comprises multiple characters; performing grouping processing on the tuple data to obtain multiple sub-tuple data; each of the sub-tuple data comprises at least one character; adding a grouping identifier to each of the sub-tuple data to obtain multiple initial identifier sub-tuple data; performing flow table query on each of the initial identifier sub-tuple data based on a pre-configured flow table rule to obtain a query result of each of the initial identifier sub-tuple data and a priority corresponding to the query result; and sequentially executing action information corresponding to each of the query results based on the priority.

Description

Technical Field

[0001] This application relates to the field of network communication technology, and includes, but is not limited to, a data processing method, apparatus, device, and storage medium. Background Technology

[0002] For existing cloud computing virtualization technologies, open-source distributed multilayer virtual switches (OpenvSwitch) are commonly used to implement virtual switch functions. Currently, smart network interface cards (NICs) are gradually offloading some OpenvSwitch functions to Field-Pogrammable Gate Array (FPGA) hardware for accelerated processing. In the hardware implementation process, the most important step is flow table rule lookup.

[0003] In related technologies, virtual switches have high requirements for latency (the time difference between executing the first lookup request and obtaining the first lookup result) and query efficiency (the number of lookup results that can be completed per second) during virtual implementation. Because common flow table queries require multiple data processing and table lookup operations, the resulting latency is significant, making it impossible to reduce resource consumption in the initial data processing stage.

[0004] Therefore, improving data processing efficiency is a technical problem that must be addressed. Summary of the Invention

[0005] In view of the problems in related technologies, this application provides a data processing method, apparatus, device and storage medium.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] This application provides a data processing method, the method comprising:

[0008] In response to a business request, feature extraction is performed on the data packets in the business request to obtain tuple data; the tuple data includes multiple characters;

[0009] The tuple data is grouped to obtain multiple sub-tuple data; each sub-tuple data includes at least one character.

[0010] Add a grouping identifier to each of the sub-tuple data to obtain multiple initial identifier sub-tuple data;

[0011] Based on pre-configured flow table rules, flow table queries are performed on each of the initial identifier sub-tuple data to obtain the query results for each of the initial identifier sub-tuple data and the priority corresponding to the query results;

[0012] Based on the priority, the action information corresponding to each query result is executed sequentially.

[0013] This application provides a data processing apparatus, the apparatus comprising:

[0014] The feature extraction module is used to extract features from the data packets in the business request in response to the business request, and obtain tuple data; the tuple data includes multiple characters;

[0015] A grouping module is used to group the tuple data to obtain multiple sub-tuple data; each sub-tuple data includes at least one character.

[0016] An add module is used to add a grouping identifier to each of the sub-tuple data to obtain multiple initial identifier sub-tuple data;

[0017] The query module is used to perform flow table queries on each of the initial identifier sub-tuple data based on pre-configured flow table rules, and obtain the query results of each of the initial identifier sub-tuple data and the priority corresponding to the query results;

[0018] The execution module is used to execute the action information corresponding to each query result sequentially based on the priority.

[0019] In some embodiments, the query module is further configured to query the query results of each initial identifier sub-tuple data and the priority corresponding to the query results from preset table entry cache units and result cache units based on pre-configured flow table rules.

[0020] In some embodiments, the apparatus further includes: a storage module, configured to receive flow table configuration information; the flow table configuration information includes tuple data to be configured, fuzzy operation position, operation result, and priority index information; perform information processing on the tuple data to be configured and the fuzzy operation position to obtain the content of the table entry to be looked up; write the content of the table entry to be looked up into the table entry cache unit of the flow table rule; and store the operation result into the result cache unit according to the priority index corresponding to the priority index information.

[0021] In some embodiments, the query module is further configured to perform a flow table query on each of the initial identifier sub-tuple data based on pre-configured flow table rules in the FPGA's internal running memory.

[0022] In some embodiments, the query module is further configured to perform parallel queries on the plurality of initial identifier sub-tuple data to obtain a plurality of initial query results; and to perform AND operation processing on the plurality of initial query results to obtain at least one query result and a priority corresponding to each query result.

[0023] In some embodiments, the query module is further configured to concatenate the table entry address in the table entry cache unit with the reuse identifier to obtain the reuse table entry address; write the reuse table entry address into the table entry corresponding to the table entry address of the flow table rule to obtain the reuse flow table rule; and perform flow table query on the sub-tuple data based on the reuse flow table rule.

[0024] In some embodiments, the execution module is further configured to: when a corresponding query result is found in the internal running memory of the FPGA, directly execute the action information corresponding to the query result; when no corresponding query result is found in the internal running memory of the FPGA, perform hash calculation on the tuple data to obtain a hash value; and perform a flow table query on the hash value based on the flow table rules to obtain the query result of the hash value and the priority corresponding to the query result.

[0025] This application provides a data processing device, including: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the above-described data processing method.

[0026] This application provides a computer-readable storage medium storing executable instructions, which are used to cause a processor to execute the executable instructions to implement the above-described data processing method.

[0027] The data processing method, apparatus, device, and storage medium provided in this application first extract features from data packets in a service request to obtain tuple data; then, the tuple data is grouped to obtain multiple sub-tuple data; next, a grouping identifier is added to each sub-tuple data to obtain multiple initial identifier sub-tuple data; based on pre-configured flow table rules, a flow table query is performed on each initial identifier sub-tuple data to obtain the query result of each initial identifier sub-tuple data and the priority corresponding to the query result; finally, based on the priority, the action information corresponding to each query result is executed sequentially. Thus, this application embodiment enables the direct grouping of tuple data with larger characters to obtain sub-tuple data with smaller characters during flow table queries. Subsequent flow table queries using these sub-tuple data save significant data computation time, effectively reduce latency caused by computation, and improve data processing speed, thereby enhancing system performance. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the implementation method of SDN cloud network based on FPGA acceleration card;

[0029] Figure 2 This is a flowchart illustrating a data forwarding method for an FPGA-based smart network interface card.

[0030] Figure 3 This is a schematic diagram of the structure of the data processing system provided in the embodiments of this application;

[0031] Figure 4 This is a schematic flowchart of a data processing method provided in an embodiment of this application;

[0032] Figure 5 This is another schematic flowchart of the data processing method provided in the embodiments of this application;

[0033] Figure 6 This is a schematic diagram of the data processing architecture provided in the embodiments of this application;

[0034] Figure 7 This is a schematic diagram of the tuple data processing architecture provided in the embodiments of this application;

[0035] Figure 8 This is a flowchart illustrating the tuple data processing method provided in an embodiment of this application;

[0036] Figure 9 This is a schematic diagram of tuple data grouping provided in an embodiment of this application;

[0037] Figure 10 This is a flowchart illustrating the flow table rule configuration method provided in an embodiment of this application;

[0038] Figure 11 This is a schematic diagram of the flow table rule structure provided in an embodiment of this application;

[0039] Figure 12 This is a schematic diagram of the lookup result structure provided in the embodiments of this application;

[0040] Figure 13 This is a schematic diagram of the composition structure of the data processing apparatus provided in the embodiments of this application;

[0041] Figure 14 This is a schematic diagram of the composition structure of the data processing device provided in the embodiments of this application. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this application pertain. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit the application.

[0044] Here, we will first explain the technical terms involved in the embodiments of this application.

[0045] (1) Hash computation is a computation that transforms an input of arbitrary length (also called a pre-image) into a fixed-length output using a hash algorithm. This output is the hash value. This transformation is a compression mapping, meaning that the space of hash values ​​is usually much smaller than the space of inputs. Different inputs may hash to the same output, so it is impossible to determine a unique input value from the hash value. Simply put, it is a function that compresses a message of arbitrary length into a message digest of a certain fixed length.

[0046] (2) Software-defined network (SDN) is a new type of innovative network architecture that can define and control the network through software programming. The control plane and forwarding plane of software-defined network have the characteristics of separation, openness and programmability.

[0047] (3) Random Access Memory (RAM) is an internal memory that can directly exchange data with the Central Processing Unit (CPU).

[0048] In related technologies, an SDN cloud network implementation method based on an FPGA accelerator card includes: when a network data packet enters a network device, the network data packet is forwarded to the FPGA accelerator card; after parsing the network data packet, the FPGA accelerator card performs OpenFlow flow table rule matching on the network data packet; if a match is found, the FPGA accelerator card executes the matched OpenFlow flow table rule action and forwards it to the network device; if no match is found, the kernel notifies the user space to send the data packet to the virtual machine control process; when the control process queries the user space for OpenFlow flow table rule matching, the flow table is sent to the FPGA accelerator card to execute the matched OpenFlow flow table rule action and forwards it to the network device; after the step of sending the data packet to the virtual machine control process, the method further includes the step of: when the control process queries the user space for OpenFlow flow table rules and finds no match, the network data packet is sent to the SDN controller in the form of a data packet; the SDN controller calculates the OpenFlow flow table and sends it to the virtual switch control process; the virtual switch control process sends the OpenFlow flow table to the FPGA accelerator card according to the caching rules; the FPGA accelerator card executes the OpenFlow flow table rule action and forwards it to the network device. Figure 1 This is a flowchart illustrating the implementation method of SDN cloud network based on FPGA acceleration card, such as... Figure 1 As shown, the method includes the following steps: S11, flow table configuration; S12, tuple data calculation; S13, calculation result lookup; S14, lookup result processing.

[0049] In related technologies, a data forwarding method for an FPGA-based smart network interface card (NIC) includes: defining a module on the FPGA of the smart NIC for storing high-frequency flow table rules as a first cache module, and a hardware module on the smart NIC for storing low-frequency flow table rules as a second cache module. High-frequency flow table rules are those that are hit by data packets within a set time period, and low-frequency flow table rules are those that are not hit by data packets within the set time period. When a data packet enters the smart NIC, the system queries the first cache module to determine whether the data packet hits a flow table rule in the first cache module. If yes, a specified action is executed, and the hit counter in the first cache module is incremented by 1. The specified action includes any one of forwarding, dropping, mirroring, or adding encapsulation. If no, the system queries the second cache module to determine whether the data packet hits a flow table rule in the second cache module. If yes, a specified action is executed, and the hit counter in the second cache module is incremented by 1. If no, the system on a network interface card (SOC) of the smart NIC... The CPU determines the action to be performed by the data packet by querying its internal flow table rules, and sends the flow table rules matched by the data packet to the first cache module. It determines the popularity of the current flow table rule by periodically polling, and switches the current flow table rule between the first cache module and the second cache module according to the popularity of the current flow table rule. Figure 2 This is a flowchart illustrating a data forwarding method for an FPGA-based smart network interface card (NIC). Figure 2 As shown, the method includes the following steps: S21, flow table configuration; S22, tuple data calculation; S23, calculation result lookup in table 1; S24, calculation result lookup in table 2; S25, lookup result processing. This method does not optimize the original lookup algorithm and cannot solve the latency caused by multiple hash calculations. It simply distinguishes between hot and cold flow table entries and still requires multiple accesses to the second cache module, continuously moving frequently accessed data into the first cache module. Furthermore, it consumes significant resources for computational processing to collect popularity information. In contrast, by improving the flow table lookup structure and storing frequently accessed flow tables in the first cache module, performance is improved by reducing the access frequency of the second cache module.

[0050] In related technologies, multiple hash calculations in tuple data processing result in significant time consumption, and the use of external DDR memory for result lookup leads to substantial latency. Furthermore, reading data from random addresses is inefficient. Therefore, the lookup operation suffers from high latency and significantly impacts system performance.

[0051] Although fuzzy matching is a common basic function, traditional fuzzy matching and regular exact matching are not processed separately, but are processed in the same lookup table structure. Since the flow table related to fuzzy matching is used more frequently than the regular matching table, the number of external DDR accesses can be greatly reduced by implementing the fuzzy matching function in the FPGA's on-chip RAM.

[0052] To address at least one of the aforementioned problems in related technologies, this application provides a data processing method. First, features are extracted from data packets in a business request to obtain tuple data. Then, the tuple data is grouped to obtain multiple sub-tuple data. Next, a grouping identifier is added to each sub-tuple data to obtain multiple initial identifier sub-tuple data. Based on pre-configured flow table rules, a flow table query is performed on each initial identifier sub-tuple data to obtain the query result and its corresponding priority. Finally, based on the priority, the action information corresponding to each query result is executed sequentially. Thus, this application enables the direct grouping of larger tuple data into smaller sub-tuple data during flow table queries. Subsequent flow table queries using these sub-tuple data save significant data computation time, effectively reduce latency caused by computation, and improve data processing speed, thereby enhancing system performance.

[0053] The following describes exemplary applications of the data processing device according to embodiments of this application. The data processing device provided in this application can be implemented as a terminal or as a server. In one implementation, the data processing device provided in this application can be implemented as various types of terminals such as laptops, tablets, desktop computers, and mobile devices. In another implementation, the data processing device provided in this application can also be implemented as a server. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application embodiment. The following will describe exemplary applications when the data processing device is implemented as a server.

[0054] join Figure 3 , Figure 3This is a schematic diagram of the data processing system provided in an embodiment of this application. To process data, this embodiment of the application can provide a data processing platform, which can be implemented as a data processing application. The data processing system 30 provided in this embodiment of the application includes a terminal 100, a network 200, and a server 300, wherein the server 300 is the server for the data processing application. The terminal 100 is connected to the server 300 through the network 200, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0055] In some embodiments, please refer to Figure 3 During data processing, terminal 100 sends its initiated service request to server 300. In response to the service request, server 300 extracts features from the data packets in the request to obtain tuple data, each containing multiple characters; it then groups the tuple data to obtain multiple sub-tuple data; each sub-tuple data contains at least one character; a grouping identifier is added to each sub-tuple data to obtain multiple initial identifier sub-tuple data; based on pre-configured flow table rules, a flow table query is performed on each initial identifier sub-tuple data to obtain the query result and its corresponding priority; based on the priority, the action information corresponding to each query result is executed sequentially. After executing the action information corresponding to the query result, server 300 sends the execution result to terminal 100 via network 200, thus responding to the terminal's service request.

[0056] This application provides a data processing method that can be executed by a server. See also... Figure 4 , Figure 4 This is a schematic flowchart of a data processing method provided in an embodiment of this application, which will be combined with... Figure 4 The steps shown are explained.

[0057] Step S401: In response to a service request, feature extraction is performed on the data packets in the service request to obtain tuple data; the tuple data includes multiple characters.

[0058] In some embodiments, a service request may be a virtual request initiated by a user, a virtual request initiated by a terminal based on the current storage resources of the server, or a virtual request initiated by the server based on a scheduled task. The virtual request is used to request the server to dummy a virtual machine, an application, or storage.

[0059] In some embodiments, tuple data may be a local area network address (MAC, Media Access Control Address), an Internet Protocol address (IP, Internet Protocol Address), a port number, etc.

[0060] Step S402: The tuple data is grouped to obtain multiple sub-tuple data; each sub-tuple data includes at least one character.

[0061] In some embodiments, the server can group tuple data according to a preset ratio to obtain multiple sub-tuple data. For example, tuple data can be a string of 1000 bits, which can be divided into 4 groups in a 1:1:1:1 ratio to obtain 4 groups of characters, each with 250 bits, namely sub-tuple data 1, sub-tuple data 2, sub-tuple data 3, and sub-tuple data 4.

[0062] Step S403: Add a grouping identifier to each of the sub-tuple data to obtain multiple initial identifier sub-tuple data.

[0063] In some embodiments, the group identifier can distinguish different sub-tuple data. The group identifier can be a number with a specific rule or a letter with a specific rule. For example, the group identifier can be 000001, 000002, ..., 00000N, etc., or the group identifier can also be A1, B2, C3, D4, etc.

[0064] In this embodiment of the application, the server can add a grouping identifier to the grouped sub-tuple data to obtain multiple initial identifier sub-tuple data.

[0065] Step S404: Based on the pre-configured flow table rules, perform a flow table query on each of the initial identifier sub-tuple data to obtain the query result of each of the initial identifier sub-tuple data and the priority corresponding to the query result.

[0066] In some embodiments, a flow table rule refers to a lookup rule, which includes multiple table entries. Each table entry includes at least one of the following: tuple data, the lookup result corresponding to the tuple data, and the priority corresponding to the lookup result. The server can perform a flow table query on the extracted tuple data to find the corresponding lookup result, and then process the business request accordingly based on the priority corresponding to the lookup result, that is, perform the corresponding operation according to the action information corresponding to the query result.

[0067] Step S405: Based on the priority, execute the action information corresponding to each query result in sequence.

[0068] In some embodiments, the action information may be information corresponding to processing actions such as forwarding, dropping, mirroring, and adding encapsulation.

[0069] Therefore, the data processing method provided in this application first extracts features from the data packets in the business request to obtain tuple data; then, it groups the tuple data to obtain multiple sub-tuple data; next, it adds a grouping identifier to each sub-tuple data to obtain multiple initial identifier sub-tuple data; based on pre-configured flow table rules, it performs a flow table query on each initial identifier sub-tuple data to obtain the query result of each initial identifier sub-tuple data and the priority corresponding to the query result; finally, based on the priority, it executes the action information corresponding to each query result in sequence. Thus, this application embodiment enables the direct grouping of tuple data with larger characters to obtain sub-tuple data with smaller characters during flow table queries. Subsequent flow table queries using these sub-tuple data save significant data computation time, effectively reduce latency caused by computation, and improve data processing speed, thereby enhancing system performance.

[0070] In some embodiments, step S404 may further include the following: based on pre-configured flow table rules, querying the query results of each initial identifier sub-tuple data and the priority corresponding to the query results from the preset table entry cache unit and result cache unit.

[0071] In some embodiments, before performing a flow table query on the initial identifier sub-tuple data, flow table rules can be obtained through flow table configuration information. Based on the foregoing embodiments, this application provides a data processing method that can be executed by a server. Figure 5 This is another schematic flowchart of the data processing method provided in the embodiments of this application, such as... Figure 5 As shown, it will be combined Figure 5 The steps shown are explained.

[0072] Step S501: The server receives the flow table configuration information.

[0073] Here, the flow table configuration information includes the tuple data to be configured, the location of the fuzzy operation, the operation result, and the priority index information.

[0074] In step S502, the server processes the configuration tuple data and the fuzzy operation position to obtain the content of the table item to be looked up.

[0075] In step S503, the server writes the content of the table entry to be queried into the table entry cache unit of the flow table rule.

[0076] In step S504, the server stores the operation result in the result cache unit according to the priority index corresponding to the priority index information.

[0077] In some embodiments, step S404 may further include the following: performing a flow table query on each initial identifier sub-tuple data based on pre-configured flow table rules in the FPGA's internal running memory.

[0078] In some embodiments, step S404 can also be implemented as follows: First, the server performs parallel queries on multiple initial identifier sub-tuple data to obtain multiple initial query results. Then, the server performs an AND operation on the multiple initial query results to obtain at least one query result and a priority corresponding to each query result.

[0079] In practical applications, when performing an AND operation, if the two values ​​are the same, the result is the same as both values; if the two values ​​are different, the result is 0. When performing an OR operation, the result is 1 as long as either of the two values ​​is 1.

[0080] In some embodiments, the server can perform an AND operation on multiple initial query results, that is, prioritize multiple initial table lookup results to obtain the highest priority result, and the corresponding processing result is obtained by the cache module based on the highest priority query result to complete the table lookup.

[0081] In some embodiments, the above data processing method further includes the following:

[0082] First, the server concatenates the entry address in the entry cache unit with the reuse identifier to obtain the reused entry address. Then, the server writes the reused entry address into the entry corresponding to the entry address of the flow table rule, thus obtaining the reused flow table rule. Finally, based on the reused flow table rule, the server performs a flow table lookup on the sub-tuple data.

[0083] In some embodiments, the server performs flow table queries on the sub-tuple data based on the reuse flow table rules to obtain the query results corresponding to the sub-tuple data and the priorities associated with those results. Then, based on the priorities, the server executes the action information corresponding to the query results.

[0084] In some embodiments, by adding a reuse identifier, the server can reuse the table entry addresses in the table entry cache unit in different time periods to obtain the reuse flow table rules. This can reduce RAM resource usage by a factor of two. By changing the reuse identifier at different times, the server can perform reuse lookup of sub-tuple data after different groups. In this way, the server can save the FPGA's internal RAM usage by performing flow table lookup through table entry reuse.

[0085] In some embodiments, after performing step S404 above, the data processing method may further include the following:

[0086] On the one hand, when the server finds the corresponding query result in the FPGA's internal running memory, it directly executes the action information corresponding to the query result.

[0087] On the other hand, when the server does not find the corresponding query result in the FPGA's internal memory, it performs a hash calculation on the tuple data to obtain a hash value. Based on flow table rules, the server performs a flow table lookup on the hash value to obtain the query result and its corresponding priority. Then, based on the priority of the hash value's query result, it executes the action information corresponding to the query result.

[0088] In some embodiments, the hash calculation of tuple data can be a single hash calculation or multiple hash calculations. The specific number of hash calculations is determined by the query results; that is, the server stops hashing the tuple data once the corresponding query result is found. Therefore, the query result for the hash value can be multiple query results or a single query result.

[0089] The data processing method provided in this application improves the performance of the data processing system by eliminating the need for multiple hash calculations before table lookup. Tuple data is grouped, and a reuse identifier is added for direct table lookup. To further enhance processing speed, priority results are cached in a result cache unit, requiring only the extraction of the result with the highest bit set to 1 to obtain the priority result. Simultaneously, the server can directly obtain query results for all fuzzy matching entries. When adding corresponding flow table rules, priority information is simply written to the appropriate priority column, enabling flexible expansion and modification of flow table rules. Furthermore, by distinguishing between fuzzy matching and precise matching functions and processing them separately, and by moving the fuzzy matching function from external DDR memory to the FPGA's internal cache, latency is reduced and performance is improved.

[0090] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0091] This application provides a data processing method that eliminates the need for hash calculations on the tuple data of flow table rules, thereby reducing the latency caused by hash calculations. At the same time, the fuzzy matching function and the precise matching function are designed separately. By implementing the fuzzy matching function in the FPGA's on-chip RAM, the number of external DDR accesses can be greatly reduced, improving the flow table query speed.

[0092] On the one hand, the flow table rule structure in the relevant technology is optimized. First, the priority of the query result is translated through the table lookup result, so that a single query can obtain multiple query results and their priorities. This reduces the priority processing time and lowers latency. Second, by adding a reuse identifier, the table entries corresponding to the table entries of the flow table rules are reused for lookup, saving FPGA internal RAM usage. Furthermore, the multiple related flow table configuration results in the second cache module are merged, and the merged flow table configuration result is used for fuzzy matching. In the first cache module, multiple precise lookup results can be achieved by configuring fuzzy matching flow table rules once.

[0093] On the other hand, by adjusting the internal cache size, fuzzy matching rules of different scales can be supported, achieving a more efficient matching algorithm compared to the original precise lookup. This solution reduces the number of flow table configurations, reduces cache resource consumption, reduces the workload of flow table configuration, lowers power consumption, and improves performance. Compared to traditional flow table queries or hot-based methods, it reduces the number of flow table configurations. Furthermore, it eliminates the need for secondary calculations, allowing flow table queries to be performed directly using tuple data, reducing latency and improving performance.

[0094] This application provides a data processing architecture. Figure 6 This is a schematic diagram of the data processing architecture of an embodiment of this application, as shown below. Figure 6 As shown in the diagram, this application improves caching performance by adding a dedicated fuzzy matching processing module (shown in the dashed box) to the common architecture. Figure 6 The diagram shows an FGPA module 601, an external cache module 602, a cloud host 603, a high-speed network interface 604, and an embedded processor SOC 605. The FGPA module 601 includes: a Peripheral Component Interconnect Express (PCIE) port 6011, a data processing logic module 6012, a matching data processing module 6013, a flow table configuration module 6014, and a cache lookup module 6015.

[0095] Here, the flow table configuration module 6014 is used to process the key (i.e., tuple data in this embodiment), mask (i.e., action information in this embodiment), and result (i.e., query result in this embodiment) configured by the external processor into table entries to obtain optimized table entry data. The cache lookup module 6015 is used to cache table entry data and perform result lookup for the tuple data to be searched. The matching data processing module 6013 is used to extract the multiple reused lookup results and the priority results corresponding to the query results obtained from the lookup results to obtain the final table lookup result.

[0096] This application provides a tuple data processing architecture. Figure 7 This is a schematic diagram of the tuple data processing architecture provided in the embodiments of this application, as shown below. Figure 7 As shown, the tuple data processing architecture includes the following: tuple data module 701, tuple data processing module 702, exact matching flow table module 703, exact flow table configuration module 704, fuzzy matching flow table module 705, fuzzy flow table configuration module 706, and lookup result processing module 707.

[0097] In this embodiment, tuple data can be processed by inserting the data processing modules shown in the dashed box (i.e., fuzzy matching flow table module 705 and fuzzy flow table configuration module 706) based on related technologies. When the server finds the query result corresponding to the tuple data in the tuple data module, the above can be skipped. Figure 1 The processing in steps S11 and S12 (i.e., tuple data processing module 702, exact matching flow table module 703, and exact flow table configuration module 704) continues. If the server does not find a query result corresponding to the tuple data in the tuple data processing module, it will continue to wait. Figure 7 The search results for the left-hand structure.

[0098] In addition, the tuple data processing module does not need to perform the above. Figure 1 The hash calculation in step S12 saves the original hash calculation time. Secondly, the original external fuzzy matching function is moved to the internal fast memory, saving search time. Moreover, the data processing speed of the tuple data processing module is also higher than that of the existing technology. If the search result of the tuple data processing module is true, the original search step is skipped and the process directly proceeds to step four. This can reduce latency, improve processing speed, and achieve higher data packet processing speed.

[0099] The architecture provided in this application implements fuzzy matching lookup table operations using the FPGA's internal high-speed RAM, while simplifying the original lookup operation. It can directly use existing data for searching, eliminating the need for hash calculations. Furthermore, the random address lookup in the internal RAM does not affect efficiency, allowing for faster result retrieval. With proper table entry configuration, system performance can be significantly improved. Additionally, to reduce external memory reads, this application utilizes the FPGA's internal high-speed RAM for auxiliary lookup. If a corresponding result is found in the high-speed RAM, the hash calculation and external memory lookup process are skipped; otherwise, the original lookup path is followed.

[0100] Regarding the transfer of internal and external RAM, the embodiments of this application do not require the transfer of external memory content. The content searched in the internal and external memory is different. The internal RAM is used for fuzzy matching rule search, while the external memory is used for precise matching content search. A single fuzzy matching rule can achieve multiple precise matching functions. Multiple similar precise matching rules are merged into the internal RAM for search, so there is no need to transfer them, nor can it be achieved simply by transferring them.

[0101] This application provides a tuple data processing method. Figure 8 This is a flowchart illustrating the tuple data processing method provided in the embodiments of this application, as shown below. Figure 8 As shown, the tuple data processing method includes the following steps:

[0102] Step S801: The server performs the first grouping: the tuple data is grouped to obtain sub-tuple data.

[0103] In some embodiments, the tuple data is grouped into M groups, each containing L bits of sub-tuple data. After grouping the tuple data, a multiplexing identifier is added, which is used to multiplex the M entries. In practical applications, this step is time-efficient.

[0104] In step S802, the server performs a second grouping: after the sub-tuple data and the multiplexing identifier are recombined, the RAM address is obtained.

[0105] In practical applications, this step does not take much time.

[0106] Step S803: M group of table entries are looked up, and table entries are reused M times to obtain the query results.

[0107] In practical applications, each reuse of an entry in this step takes 1 clk and has a delay of 2 clk.

[0108] In step S804, the server performs priority processing on the query results to obtain the priority results corresponding to the query results.

[0109] In practical applications, this step takes 1 clk and has a delay of 10 clk.

[0110] In step S805, the server executes the action information corresponding to each query result sequentially based on the priority result.

[0111] In practical applications, this step takes 1 clk and has a delay of 10 clk.

[0112] Step S806: The server reads the lookup result.

[0113] In practical applications, this step takes 1 clk and has a delay of 2 clk.

[0114] Therefore, the tuple data processing method provided in this application embodiment groups the tuple data to be queried, and then performs parallel queries after grouping. Here, the tuple data can be divided into L bit * M groups according to a preset ratio. After grouping, the L bit sub-tuple data can be used as the table entry address to directly look up the table, obtaining a lookup result of M * N bits. Then, an AND operation is performed based on the query results of the M groups, representing N flow table query results. Afterwards, the lookup results are prioritized to obtain the highest priority result. The cache module obtains the corresponding processing result based on the highest priority query result, completing the table lookup. The tuple data processing method provided in this application embodiment, through FPGA parallel flow tables, can reach speeds several times faster than the original DDR table lookup, depending on the FPGA operating clock. When implementing OpenvSwitch in the FPGA in this application embodiment, server resource consumption needs to be considered. However, FPGA resources are limited. Therefore, this application embodiment adds a reuse identifier after the first grouping, and then reassembles the reuse identifier with the subkey (i.e., the sub-tuple data in the above embodiment) for lookup. At the same time, internal RAM resources are saved by reusing the lookup flow table. Therefore, compared with the existing technology of simple grouping, this application can be implemented by FPGA, thus saving more FPGA resources.

[0115] To facilitate understanding the above Figure 8 The tuple data grouping method described in the previous article is illustrated in the following embodiment of the present application: a schematic diagram of tuple data grouping. Figure 9 This is a schematic diagram of tuple data grouping provided in the embodiments of this application, such as... Figure 9 As shown in the figure, example diagrams are shown after the first grouping and the second grouping, such as the first grouping 901 and the second grouping 902.

[0116] First, all 224-bit tuple data is divided into 4 groups, and a 2-bit grouping multiplexing identifier is added. Then, the grouped tuple data and the multiplexing identifier are recombined into 8*9-bit sub-tuple data. In practical applications, the resource utilization is highest when the RAM in a common FPGA is configured with 9-bit addresses. In this case, multiplexing the 8 groups of entries in 4 lookups is sufficient to complete the lookup result of the 224-bit tuple data. The first grouping is for multiplexing the lookup table, and the second grouping and recombining with the multiplexing identifier aims to maximize the utilization of the FPGA's internal RAM resources. Adding a 2-bit multiplexing identifier enables four lookup entries with multiplexing, optimizing resources while ensuring lookup speed. When the system clock is operating at 400MHz, the lookup speed can reach 100 million packets per second, achieving a balance between performance and resources.

[0117] In this embodiment of the application, according to this grouping multiplexing scheme, 8 RAM blocks can be used to multiplex the result 4 times to obtain the 224-bit key lookup. If not multiplexed, approximately 25 RAM blocks are required.

[0118] This application provides a method for configuring flow table rules. Figure 10 A flowchart illustrating the flow table rule configuration method provided in this application embodiment is shown below. Figure 10 As shown in the flowchart, the flow table rule configuration method includes the following steps:

[0119] Step S1001: Receive flow table configuration information.

[0120] Step S1002: Process the data of the configuration tuple and the position of the fuzzy operation to obtain the content of the table item to be looked up.

[0121] Step S1003: Write the content of the table entry to be queried into the table entry cache unit of the flow table rule.

[0122] Step S1004: Store the operation result in the result cache unit according to the priority index corresponding to the priority index information.

[0123] In some embodiments, the table entry configuration is implemented by an external processor, i.e., an FPGA chip. The FPGA sends the tuple data to be configured, the fuzzy operation position, the operation result, and the priority index information to the table entry cache unit according to actual needs. The table entry cache unit processes the tuple information and the fuzzy operation position to obtain the content of the table entry to be queried. Then, the content of the table entry to be queried is written into the table entry cache unit of the flow table rule. According to the priority index corresponding to the priority index information, the operation result is stored in the result cache unit.

[0124] Therefore, the flow table rule configuration method provided in this application embodiment can directly obtain the query results for all fuzzy matching entries. When it is necessary to add a corresponding flow table rule, only priority information needs to be written in the appropriate priority column, realizing flexible expansion and modification of flow table rules. In addition, by distinguishing between fuzzy matching and precise matching functions and processing them separately, and by moving the fuzzy matching function from the external DDR memory lookup function to the FPGA internal cache, latency is reduced and performance is improved.

[0125] To facilitate understanding the above Figure 10 The flow table rule configuration method described in the previous article, and the present application embodiment provides a schematic diagram of the flow table rule structure. Figure 11 This is a schematic diagram of the flow table rule structure provided in an embodiment of this application, such as... Figure 11 As shown in the figure, the L / 7 group entries are illustrated.

[0126] In this embodiment, the first column identifies the concatenated value of the tuple data and the multiplexing identifier, which is also the 9-bit address of this table entry. Each subsequent column represents the matching result between the current address (i.e., the value after concatenating the tuple data with the multiplexing identifier) ​​and the flow table rule (where 1 indicates a match and 0 indicates no match). The first column has the highest priority for matching results, and the last column has the lowest priority. A single table can reuse multiplexing identifiers, and multiple tables can perform parallel lookups of L / 7 sets of results.

[0127] In addition, by concatenating the table entry address with the multiplexing identifier, a multiplexing flow table rule is obtained, which can save FPGA internal RAM resources.

[0128] Typically, FPGA internal RAM space is quite limited. To fully utilize the internal cache, as shown in the figure above, this application's embodiment optimizes the cache entries as follows: By adding a reuse flag, the lookup table is time-division multiplexed, significantly reducing RAM resource consumption. By changing the reuse flag at different times, different groups of tuple data are reused for fuzzy matching lookup table lookups. By grouping tuple data, internal RAM utilization is improved, and less space is wasted when the tuple data is not an integer multiple of the RAM address width. The resource requirements for table entry configuration can be optimized by grouping and reuse. When reusing table entries, a reuse flag is added. The reuse flag + the total bit width of a single group of tuple data = the highest address bit width configurable by a single FPGA RAM block, thus maximizing the efficiency of using a single RAM block. The actual function of fuzzy matching is that multiple tuple data correspond to one operation result. Therefore, a single flow table will correspond to multiple actual query tuple data, and the tuple data serves as the query address. That is, the corresponding multiple address lookup results are all true, thus realizing the fuzzy matching function. At the same time, compared with traditional flow table configuration, table entry storage space is saved. The table entry results include priority processing results. When configuring the table entry, the priorities are sorted from high to low based on the priority index information and written into the flow table rules.

[0129] To facilitate understanding of the above flow table rule structure, this application provides a schematic diagram of the lookup result structure. Figure 12 This is a schematic diagram of the lookup result structure provided in the embodiments of this application, such as... Figure 12 As shown in the figure, two types of lookup results are presented: 1 indicates a match, 0 indicates a non-match, and priority levels are also shown, with the leftmost being the highest priority and the rightmost being the lowest priority.

[0130] Therefore, while reusing lookup results increases the time required for a single query and reduces the overall performance of the lookup operation, it can significantly reduce the resource consumption of the internal cache RAM, enabling more fuzzy matching flow table queries with fewer resources. Furthermore, when configuring multiple flow table rules, this can be achieved by horizontally expanding RAM connections and sharing address lines. This flexible approach also improves RAM utilization efficiency, achieving resource savings. The result with the highest bit set to 1 is extracted from the query result, indicating the corresponding rule match. The corresponding rule processing action information is then retrieved from the query result cache to complete a single query.

[0131] The data processing method provided in this application adds an internal fuzzy matching flow table cache structure, processing fuzzy matching operations separately, which can significantly improve performance and reduce latency. The existing table lookup algorithm has been upgraded, reducing time consumption, latency, and the workload of external processors (HOST or SOC) (multiple flow tables are merged through fuzzy matching, eliminating the need to repeatedly move hot tables and significantly reducing external memory read time). The method saves computational mapping time and speeds up processing by directly using grouped tuple data for table lookup without mapping and computation. To address the fuzzy matching priority issue, the lookup results are prioritized, which increases the complexity of subsequent calculations and significantly impacts data processing speed. A simple method of sorting priorities from highest to lowest, placing the highest priority in the highest bit, makes processing faster and more convenient. This method meets the needs of application scenarios with numerous similar flow tables and complex configurations within a local area network, reducing operational costs.

[0132] Figure 13 This is a schematic diagram of the composition structure of the data processing device provided in the embodiments of this application, as shown below. Figure 13 As shown, the data processing device 1300 includes: a feature extraction module 1301, used to extract features from data packets in a business request in response to a business request to obtain tuple data; the tuple data includes multiple characters; a grouping module 1302, used to group the tuple data to obtain multiple sub-tuple data; each sub-tuple data includes at least one character; an adding module 1303, used to add a grouping identifier to each sub-tuple data to obtain multiple initial identifier sub-tuple data; a query module 1304, used to perform a flow table query on each initial identifier sub-tuple data based on pre-configured flow table rules to obtain a query result for each initial identifier sub-tuple data and a priority corresponding to the query result; and an execution module 1305, used to execute the action information corresponding to each query result sequentially based on the priority.

[0133] In some embodiments, the query module is further configured to query the query results of each initial identifier sub-tuple data and the priority corresponding to the query results from preset table entry cache units and result cache units based on pre-configured flow table rules.

[0134] In some embodiments, the apparatus further includes: a storage module, configured to receive flow table configuration information; the flow table configuration information includes tuple data to be configured, fuzzy operation position, operation result, and priority index information; perform information processing on the tuple data to be configured and the fuzzy operation position to obtain the content of the table entry to be looked up; write the content of the table entry to be looked up into the table entry cache unit of the flow table rule; and store the operation result into the result cache unit according to the priority index corresponding to the priority index information.

[0135] In some embodiments, the query module is further configured to perform a flow table query on each of the initial identifier sub-tuple data based on pre-configured flow table rules in the FPGA's internal running memory.

[0136] In some embodiments, the query module is further configured to perform parallel queries on the plurality of initial identifier sub-tuple data to obtain a plurality of initial query results; and to perform AND operation processing on the plurality of initial query results to obtain at least one query result and a priority corresponding to each query result.

[0137] In some embodiments, the query module is further configured to concatenate the table entry address in the table entry cache unit with the reuse identifier to obtain the reuse table entry address; write the reuse table entry address into the table entry corresponding to the table entry address of the flow table rule to obtain the reuse flow table rule; and perform flow table query on the sub-tuple data based on the reuse flow table rule.

[0138] In some embodiments, the execution module is further configured to: when a corresponding query result is found in the internal running memory of the FPGA, directly execute the action information corresponding to the query result; when no corresponding query result is found in the internal running memory of the FPGA, perform hash calculation on the tuple data to obtain a hash value; and perform a flow table query on the hash value based on the flow table rules to obtain the query result of the hash value and the priority corresponding to the query result.

[0139] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment; therefore, it will not be repeated. For technical details not disclosed in this apparatus embodiment, please refer to the description of the method embodiment of this application for understanding.

[0140] It should be noted that, in the embodiments of this application, if the above-described data processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0141] Correspondingly, embodiments of this application provide a data processing device. Figure 14 This is a schematic diagram of the composition structure of the data processing device provided in the embodiments of this application, such as... Figure 14 As shown, the data processing device 1400 includes at least a processor 1401 and a computer-readable storage medium 1402 configured to store executable instructions, wherein the processor 1401 generally controls the overall operation of the data processing device. The computer-readable storage medium 1402 is configured to store instructions and applications executable by the processor 1401, and may also cache data to be processed or processed by various modules in the processor 1401 and the data processing device 1400, and may be implemented using flash memory or random access memory (RAM).

[0142] This application provides a storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the method provided in this application, for example... Figure 2 The method shown.

[0143] In some embodiments, the storage medium may be a computer-readable storage medium, such as a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disc, or a compact disk-read-only memory (CD-ROM); or it may be a device that includes one or any combination of the above-mentioned memories.

[0144] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0145] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file containing other programs or data, for example, in one or more scripts within a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files storing one or more modules, subroutines, or code sections). As an example, executable instructions may be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0146] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

[0147] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0148] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not performed.

[0149] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method includes: In response to a business request, feature extraction is performed on the data packets in the business request to obtain tuple data; the tuple data includes multiple characters; The tuple data is grouped to obtain multiple sub-tuple data; each sub-tuple data includes at least one character. Add a grouping identifier to each of the sub-tuple data to obtain multiple initial identifier sub-tuple data; Based on pre-configured flow table rules, flow table queries are performed on each of the initial identifier sub-tuple data to obtain the query results for each of the initial identifier sub-tuple data and the priority corresponding to the query results; Based on the priority, the action information corresponding to each query result is executed sequentially.

2. The method according to claim 1, characterized in that, The process of performing a flow table query on each initial identifier sub-tuple based on pre-configured flow table rules to obtain the query result for each initial identifier sub-tuple and the priority corresponding to the query result includes: Based on pre-configured flow table rules, the query results and the priority corresponding to each initial identifier sub-tuple data are queried from the preset table entry cache unit and result cache unit.

3. The method according to claim 2, characterized in that, The method further includes: Receive flow table configuration information; the flow table configuration information includes tuple data to be configured, fuzzy operation position, operation result, and priority index information; The data of the tuple to be configured and the position of the fuzzy operation are processed to obtain the content of the table item to be looked up. Write the content of the table entry to be queried into the table entry cache unit of the flow table rule; The operation result is stored in the result cache unit according to the priority index corresponding to the priority index information.

4. The method according to any one of claims 1 to 3, characterized in that, The step of performing a flow table lookup for each of the initial identifier sub-tuples includes: Based on pre-configured flow table rules, a flow table query is performed on each of the initial identifier sub-tuple data in the FPGA's internal running memory.

5. The method according to any one of claims 1 to 3, characterized in that, The process of performing a flow table query on each initial identifier sub-tuple based on pre-configured flow table rules to obtain the query result for each initial identifier sub-tuple and the priority corresponding to the query result includes: Parallel queries are performed on the multiple initial identifier sub-tuple data to obtain multiple initial query results; The multiple initial query results are subjected to an AND operation to obtain at least one query result and a priority corresponding to each query result.

6. The method according to claim 3, characterized in that, The method further includes: The table entry address in the table entry cache unit is concatenated with the reuse identifier to obtain the reuse table entry address; Write the address of the reused table entry into the table entry corresponding to the table entry address of the flow table rule to obtain the reused flow table rule; Based on the reusable flow table rules, flow table queries are performed on the sub-tuple data.

7. The method according to claim 4, characterized in that, After performing a flow table lookup on each of the initial identifier sub-tuples, the method further includes: When the corresponding query result is found in the internal running memory of the FPGA, the action information corresponding to the query result is executed directly. If no corresponding query result is found in the internal running memory of the FPGA, the tuple data is hashed to obtain a hash value. Based on the flow table rules, a flow table query is performed on the hash value to obtain the query result of the hash value and the priority corresponding to the query result.

8. A data processing apparatus, characterized in that, The device includes: The feature extraction module is used to extract features from the data packets in the business request in response to the business request, and obtain tuple data; the tuple data includes multiple characters; A grouping module is used to group the tuple data to obtain multiple sub-tuple data; each sub-tuple data includes at least one character. An add module is used to add a grouping identifier to each of the sub-tuple data to obtain multiple initial identifier sub-tuple data; The query module is used to perform flow table queries on each of the initial identifier sub-tuple data based on pre-configured flow table rules, and obtain the query results of each of the initial identifier sub-tuple data and the priority corresponding to the query results. The execution module is used to execute the action information corresponding to each query result sequentially based on the priority.

9. A data processing device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The device stores executable instructions for causing a processor to execute the executable instructions to implement the data processing method according to any one of claims 1 to 7.

Images