A fast reasoning deployment method and device for programmable switches

Abstract

The present invention relates to the field of ultra-high-speed programmable data plane technology, and discloses a fast reasoning deployment method and device for programmable switches, including: obtaining a training data set for training a neural network model; determining a first lookup table corresponding to the neural network model based on the training data set; obtaining a neural network model calculation graph corresponding to the neural network model; integrating the operators required to be quantified by the neural network into a second lookup table based on the neural network model calculation graph and the first lookup table, and deploying the second lookup table in a programmable switch; the programmable switch receives a data packet, and performs data reasoning on the data packet to obtain a reasoning result. The present invention utilizes the programming capability of the programmable switch, and while performing data packet exchange, implements a pre-trained neural network model through the line-speed flow table resources of the programmable data plane, completes the data packet reasoning task of the data plane, saves computing resources, and realizes high-speed reasoning tasks.

Description

Technical Field

[0001] The present invention relates to the fields of next generation Internet architecture, network intelligence, and ultra-high-speed programmable data plane technology, and in particular to a fast reasoning deployment method and device for programmable switches. Background Art

[0002] In recent years, with the rapid development of artificial intelligence, more and more fields are trying to use artificial intelligence to give new capabilities to related fields. Among them, with the diversified development of Internet application types and the dynamic growth of cyberspace security needs, more and more workers are trying to use artificial intelligence to empower the network field, or use the network to provide support for artificial intelligence. However, existing intelligent solutions usually sacrifice throughput and latency to meet higher flexibility, which is contrary to the high throughput and low latency characteristics of network core path switches; in addition, in order to deploy intelligent models, large computing resources are usually required, and core path switches usually have more flow tables to support table lookup forwarding, but not enough computing resources to deploy intelligent models. Summary of the invention

[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0004] To this end, an object of the present invention is to propose a fast inference deployment method for programmable switches, which utilizes the programming capabilities of programmable switches in ultra-high-speed networks. While performing data packet switching, a pre-trained neural network model is implemented through the line-speed flow table resources of the programmable data plane to complete the data packet inference task of the data plane, thereby saving computing resources and achieving high-speed inference tasks.

[0005] Another object of the present invention is to provide a fast reasoning deployment device for programmable switches.

[0006] To achieve the above object, an embodiment of the present invention provides a fast reasoning deployment method for a programmable switch, including:

[0007] Obtain a training dataset for training a neural network model;

[0008] Based on the training data set, determining a first lookup table corresponding to the neural network model;

[0009] Obtaining a neural network model calculation graph corresponding to the neural network model;

[0010] Based on the neural network model calculation graph and the first lookup table, integrating operators that need to be quantized in the neural network into a second lookup table, and deploying the second lookup table in a programmable switch;

[0011] The programmable switch receives a data packet, and performs data inference on the data packet to obtain an inference result.

[0012] The fast reasoning deployment method for a programmable switch according to an embodiment of the present invention may also have the following additional technical features:

[0013] Further, the determining, based on the training data set, a first lookup table corresponding to the neural network model comprises:

[0014] Splitting the features of the training data set into multiple feature subspaces;

[0015] An initial binary decision tree is constructed by an unsupervised decision tree algorithm, wherein each tree node in the initial binary decision tree includes a threshold value and an index value;

[0016] Taking the initial bucket corresponding to the training data on each feature subspace as the root node of the initial binary decision tree, and performing bucket division on the nodes in the initial binary decision tree until a target binary decision tree is constructed;

[0017] The mean of each index in the bucket at the leaf node of the target binary decision tree is determined as the corresponding cache centroid, and each new cache centroid is determined as the first lookup table corresponding to the neural network model.

[0018] Furthermore, the initial bucket corresponding to the training data on each feature subspace is used as the root node of the initial binary decision tree, and the nodes in the initial binary decision tree are divided into buckets until a target binary decision tree is constructed, including:

[0019] Taking the initial bucket corresponding to the training data on each feature subspace as the root node of the initial binary decision tree;

[0020] On the node, traverse each index in the bucket and determine the best split point at each target index;

[0021] Obtaining a threshold and an index value of the node based on the target segmentation point, and determining a first sub-bucket divided into the bucket as a left node of the node and a second sub-bucket as a right node of the node;

[0022] Repeat the above steps until the target binary decision tree is constructed.

[0023] Furthermore, the step of integrating the operators that need to be quantized of the neural network into a second lookup table based on the neural network model calculation graph and the first lookup table includes:

[0024] Based on the neural network model calculation graph, an operator that needs to be quantized is obtained, wherein the operator that needs to be quantized includes a linear layer, a normalization layer, and an activation layer;

[0025] Integrating the parameter matrix and bias value corresponding to the linear layer into the table entries of the first lookup table to obtain a third lookup table;

[0026] The normalization layer maps the feature vector before normalization to the corresponding table entry by applying an inverse linear transformation to the division threshold in the third lookup table to obtain a fourth lookup table;

[0027] The activation layer calculates an inverse function of the activation function, and applies the calculation result of the inverse function to the partition threshold of the fourth lookup table to obtain a second lookup table.

[0028] Furthermore, the programmable switch receives a data packet and performs data reasoning on the data packet to obtain a reasoning result, including:

[0029] The programmable switch receives a data packet from a network port, dynamically parses the data packet according to a protocol through a parser of a programmable data plane, and pre-saves the parsed feature vector required by the neural network model in metadata;

[0030] Based on the metadata, starting a matching action table pipeline of the programmable switch;

[0031] The matching action table pipeline obtains the metadata, matches the metadata with the second lookup table through a ternary content addressable memory, and performs data inference according to the matching action table pipeline based on the matching result to obtain an inference result.

[0032] Further, matching the metadata with the second lookup table through a ternary content addressable memory, and performing data reasoning according to the matching action table pipeline based on the matching result to obtain a reasoning result, includes:

[0033] Matching the metadata with the partition threshold in the second lookup table through a ternary content addressable memory, indexing to a corresponding target table entry, and retrieving a pre-cached target result from the target table entry;

[0034] The target results are summed to obtain an output vector, and the output is stored in the metadata and transmitted to the matching action table of the next stage to execute the neural network operator of the next stage. The above steps are repeated until the matching action table pipeline is completed and the inference result is obtained.

[0035] Furthermore, the method further comprises:

[0036] storing the inference result in a specified stateful register for subsequent inference; or

[0037] Processing the data packet accordingly according to the inference result; or

[0038] The inference result is encapsulated into the data packet through the reverse parser and output through the designated network port.

[0039] To achieve the above object, another embodiment of the present invention provides a fast reasoning deployment device for a programmable switch, the device comprising:

[0040] A first acquisition module is used to acquire a training data set for training a neural network model;

[0041] A determination module, configured to determine a first lookup table corresponding to the neural network model based on the training data set;

[0042] A second acquisition module is used to acquire a neural network model calculation graph corresponding to the neural network model;

[0043] A deployment module, configured to integrate operators that need to be quantized of the neural network into a second lookup table based on the neural network model calculation graph and the first lookup table, and deploy the second lookup table in a programmable switch;

[0044] The inference module is used for the programmable switch to receive a data packet and perform data inference on the data packet to obtain an inference result.

[0045] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0047] Figure 1 A flowchart of a fast reasoning deployment method for a programmable switch according to an embodiment of the present invention;

[0048] Figure 2 The figure is a schematic diagram of the structure of a fast inference deployment device for a programmable switch according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0050] The following describes a fast reasoning deployment method and device for a programmable switch proposed according to an embodiment of the present invention with reference to the accompanying drawings.

[0051] First, a fast reasoning deployment method for a programmable switch proposed according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0052] Figure 1 The present invention is a flowchart of a fast reasoning deployment method for a programmable switch according to an embodiment of the present invention.

[0053] like Figure 1 As shown, the fast reasoning deployment method for programmable switches includes the following steps:

[0054] Step S1, obtaining a training data set for training a neural network model;

[0055] Step S2, determining a first lookup table corresponding to the neural network model based on the training data set;

[0056] In one embodiment of the present invention, after obtaining the training data set through the above steps, a first lookup table corresponding to the neural network model can be determined based on the training data set.

[0057] Specifically, in one embodiment of the present invention, the method for determining the first lookup table corresponding to the neural network model based on the training data set may include the following steps:

[0058] Step S21, splitting the features of the training data set into multiple feature subspaces;

[0059] Step S22, constructing an initial binary decision tree by an unsupervised decision tree algorithm, wherein each tree node in the initial binary decision tree includes a threshold value and an index value;

[0060] Step S23, taking the initial bucket corresponding to the training data on each feature subspace as the root node of the initial binary decision tree, and dividing the nodes in the initial binary decision tree into buckets until a target binary decision tree is constructed;

[0061] Step S24, determining the mean of each index in the bucket at the leaf node of the target binary decision tree as the corresponding cache centroid, and determining each new cache centroid as the first lookup table corresponding to the neural network model.

[0062] In one embodiment of the present invention, the training data set based on machine learning and the real data set follow the assumption of independent and identical distribution. Based on this, the training data set is Learn to find the most representative subvector, called the centroid , we can learn from the real data set The centroid .

[0063] In one embodiment of the present invention, the features of the training data set can be split into feature subspaces, each of which contains features, called subvectors , training data set In the subvector All training data on .

[0064] And, in one embodiment of the present invention, an unsupervised decision tree algorithm can be used to construct a layer number of The initial binary decision tree , where each tree node in the initial binary decision tree includes a threshold and index value .

[0065] Further, in one embodiment of the present invention, after the initial binary decision tree is constructed through the above steps, a greedy strategy can be used for unsupervised parameter learning. Specifically, in one embodiment of the present invention, each training data set subspace The training data on As the initial bucket As the root node of the initial binary decision tree , the nodes in the initial binary decision tree are divided into buckets until the target binary decision tree is constructed.

[0066] In one embodiment of the present invention, the method of using the initial bucket corresponding to the training data on each feature subspace as the root node of the initial binary decision tree, and performing bucket division on the nodes in the initial binary decision tree until the target binary decision tree is constructed may include the following steps:

[0067] Step 1: Use the initial bucket corresponding to the training data on each feature subspace as the root node of the initial binary decision tree;

[0068] Step 2: traverse each index in the bucket on the node and determine the best split point at each target index;

[0069] Step 3: obtaining the threshold and index value of the node based on the target segmentation point, and determining the first sub-bucket divided into the bucket as the left node of the node and the second sub-bucket as the right node of the node;

[0070] Step 4: Repeat the above steps until the target binary decision tree is constructed.

[0071] Specifically, in one embodiment of the present invention, each training data set subspace The training data on As the initial bucket As the root node of the initial binary decision tree After that, at the i-th node Traverse the bucket Each index within , and at the jth index Select the best split point and divide the bucket into two buckets and In one embodiment of the present invention, the optimal segmentation point can be determined by minimizing the sum of square errors in each bucket after segmentation. Based on this, the sum of square errors can be used as an evaluation criterion to make the two buckets after segmentation and The sum of squared errors within is minimized, and the obtained index and , as a node The parameters of the two buckets will be obtained at the same time. and Place them at the nodes Left and right child nodes and Repeat the above steps until the target binary decision tree is constructed. .

[0072] And, in one embodiment of the present invention, after obtaining the target binary decision tree through the above steps, the leaf nodes can be Bucket The mean of each index is used as the new cache centroid , and each new cache centroid, A first lookup table corresponding to the neural network model is determined.

[0073] Step S3, obtaining a neural network model calculation graph corresponding to the neural network model;

[0074] In one embodiment of the present invention, a neural network model calculation graph corresponding to the neural network model can be obtained through the trained neural network model.

[0075] Step S4, based on the neural network model calculation graph and the first lookup table, integrating the operators that need to be quantized in the neural network into the second lookup table, and deploying the second lookup table in the programmable switch;

[0076] Among them, in one embodiment of the present invention, after determining the neural network model calculation graph and the first lookup table through the above steps, the operators that need to be quantized in the neural network can be integrated into the second lookup table based on the neural network model calculation graph and the first lookup table, and the second lookup table can be deployed in the programmable switch.

[0077] Specifically, in one embodiment of the present invention, the method for integrating the operators that need to be quantized of the neural network into the second lookup table based on the neural network model calculation graph and the first lookup table may include the following steps:

[0078] Step S41, based on the neural network model calculation graph, obtaining operators that need to be quantized, wherein the operators that need to be quantized include a linear layer, a normalization layer, and an activation layer;

[0079] Step S42, integrating the parameter matrix and bias value corresponding to the linear layer into the table items of the first lookup table to obtain a third lookup table;

[0080] Step S43, the normalization layer applies an inverse linear transformation to the thresholds divided in the cache table, maps the feature vector before normalization to the corresponding table entry, and obtains a fourth lookup table;

[0081] Step S44: The activation layer calculates the inverse function of the activation function, and applies the calculation result of the inverse function to the division threshold of the fourth lookup table to obtain the second lookup table.

[0082] In one embodiment of the present invention, the above-mentioned linear layer is a basic component in a neural network, and its operation includes matrix multiplication of an input feature vector and adding a bias value to generate an output feature vector. The corresponding partitioning threshold is calculated through the above steps, and an accurate result of the relevant centroid is obtained. In the prior art, it is necessary to add the bias value to the output vector element by element after the matrix multiplication is completed to obtain the final output vector. Based on this, in one embodiment of the present invention, the parameter matrix and the bias value can be pre-integrated into the table items of the first lookup table to obtain a third lookup table, so that the operation of adding the bias value in real time can be omitted on the data plane, which not only simplifies the calculation process of the linear layer, but also significantly improves the overall calculation efficiency by reducing the number of operations required in the data processing process.

[0083] And, in one embodiment of the present invention, the normalization layer is a key component of the neural network, and the data is standardized by applying a linear transformation to the input feature vector, making the network training process more stable and faster. Among them, the above process can multiply the input vector element by element with a vector of the same size, and add a bias vector. Based on this, in one embodiment of the present invention, a pre-calculation strategy can be adopted, by applying an inverse linear transformation to the partition threshold in the third lookup table, directly mapping the feature vector before normalization to the corresponding table entry, and obtaining a fourth lookup table, so that the output result after a series of calculation steps of normalization, matrix multiplication and bias addition can be directly obtained, and the normalization is integrated into the linear layer, without the need to recalculate each time the forward propagation is performed, which simplifies the data processing process.

[0084] Further, in one embodiment of the present invention, the activation layer performs a nonlinear transformation to increase the network's ability to handle nonlinear relationships. Similar to the above-mentioned normalization layer, the activation layer can also be optimized by a pre-calculation method, thereby accelerating the forward propagation process of the network and reducing computational complexity. In one embodiment of the present invention, the nonlinear computation requirements at runtime can be eliminated by applying pre-calculation and caching strategies. Specifically, by calculating the inverse function of the activation function and applying this inverse function to the partition threshold of the fourth lookup table, a second lookup table is obtained, thereby integrating the activation layer into the above-mentioned cache system.

[0085] In one embodiment of the present invention, for feature shape changes and pooling in a neural network, the flexibility of a programmable switch can be used to implement data format conversion and pooling processing without the need for additional design, thereby simplifying the implementation process of related operations in the neural network and maintaining processing efficiency and flexibility.

[0086] And, in one embodiment of the present invention, the operator that needs to be quantized in the neural network is integrated into the reasoning method based on the second lookup table through the above steps, and then the second lookup table is deployed in the programmable switch. In one embodiment of the present invention, the programmable switch supports integers, based on which the Power-of-two method can be used to quantize the partition threshold and table item content in the second lookup table, and the high bit is taken as the quantization result.

[0087] Step S5: The programmable switch receives the data packet and performs data inference on the data packet to obtain an inference result.

[0088] In one embodiment of the present invention, after the second lookup table is deployed in the programmable switch through the above steps, the data packet can be received through the programmable switch, and the data packet can be inferred to obtain the inference result. In one embodiment of the present invention, the programmable switch can be in an ultra-high-speed network.

[0089] Specifically, in one embodiment of the present invention, the method for the programmable switch to receive a data packet and perform data inference on the data packet to obtain an inference result may include the following steps:

[0090] Step S51, the programmable switch receives a data packet from the network port, dynamically parses the data packet according to the protocol through the parser of the programmable data plane, and pre-saves the feature vector required by the neural network model in the metadata;

[0091] Step S52, based on the metadata, starting the matching action table pipeline of the programmable switch;

[0092] Step S53, the matching action table pipeline obtains metadata, matches the metadata with the second lookup table through the ternary content addressable memory, and performs data inference according to the matching action table pipeline based on the matching result to obtain an inference result.

[0093] In one embodiment of the present invention, each data packet is assembled according to a specific protocol. After receiving the data packet from the network port, the programmable switch can dynamically parse the data packet according to the protocol through the parser of the programmable data plane, and save the feature vector required by the neural network model in the metadata in advance, and send it to the matching action table pipeline of the programmable switch. In one embodiment of the present invention, the matching action table pipeline can be pre-set by artificial experience.

[0094] And, in one embodiment of the present invention, the matching action table pipeline obtains metadata, and the metadata and the partition threshold value can be combined using a ternary content addressable memory. Matching is performed, indexing to the corresponding table entry, taking out the pre-cached result from the table entry, summing the cached results to obtain an output vector, using the vector as the result of the operation with the corresponding neural network operator as a feature, and storing the result in the metadata, transmitting it to the matching action table of the next stage to execute the neural network operator of the next stage, and repeating the above steps until the neural network inference result is obtained.

[0095] Furthermore, in one embodiment of the present invention, after obtaining the inference result through the above steps, the above method may also include: storing the inference result in a specified state register for subsequent reasoning; or, performing corresponding processing on the data packet according to the inference result; or, encapsulating the inference result into a data packet through a reverse parser and outputting it through a specified network port.

[0096] According to the fast inference deployment method for programmable switches proposed in an embodiment of the present invention, the programming capabilities of programmable switches are utilized in ultra-high-speed networks. While performing data packet switching, a pre-trained neural network model is implemented through the line-speed flow table resources of the programmable data plane to complete the data packet inference task of the data plane, thereby saving computing resources and achieving high-speed inference tasks.

[0097] Next, a fast reasoning deployment device for a programmable switch proposed according to an embodiment of the present invention is described with reference to the accompanying drawings.

[0098] Figure 2 The figure is a schematic diagram of the structure of a fast inference deployment device for a programmable switch according to an embodiment of the present invention.

[0099] like Figure 2As shown, the fast reasoning deployment device 10 for programmable switches includes: a first acquisition module 201, a determination module 202, a second acquisition module 203, a deployment module 204 and a reasoning module 205, wherein

[0100] A first acquisition module 201 is used to acquire a training data set for training a neural network model;

[0101] A determination module 202, configured to determine a first lookup table corresponding to the neural network model based on the training data set;

[0102] A second acquisition module 203 is used to acquire a neural network model calculation graph corresponding to the neural network model;

[0103] A deployment module 204 is used to integrate the operators that need to be quantized of the neural network into a second lookup table based on the neural network model calculation graph and the first lookup table, and deploy the second lookup table in the programmable switch;

[0104] The reasoning module 205 is used for the programmable switch to receive data packets and perform data reasoning on the data packets to obtain reasoning results.

[0105] Furthermore, the determination module 202 is specifically configured to:

[0106] Split the features of the training data set into multiple feature subspaces;

[0107] An initial binary decision tree is constructed by an unsupervised decision tree algorithm, wherein each tree node in the initial binary decision tree includes a threshold value and an index value;

[0108] The initial bucket corresponding to the training data on each feature subspace is used as the root node of the initial binary decision tree, and the nodes in the initial binary decision tree are divided into buckets until the target binary decision tree is constructed;

[0109] The mean of each index in the bucket at the leaf node of the target binary decision tree is determined as the corresponding cache centroid, and each new cache centroid is determined as the first lookup table corresponding to the neural network model.

[0110] Furthermore, the determination module 202 is further configured to:

[0111] Taking the initial bucket corresponding to the training data on each feature subspace as the root node of the initial binary decision tree;

[0112] On the node, traverse each index in the bucket and determine the best split point at each index;

[0113] Obtaining a threshold and an index value of the node based on the target segmentation point, and determining a first sub-bucket divided into the bucket as a left node of the node and a second sub-bucket as a right node of the node;

[0114] Repeat the above steps until the target binary decision tree is constructed.

[0115] Furthermore, the deployment module 204 is specifically used to:

[0116] Based on the computational graph of the neural network model, operators that need to be quantized are obtained, where the operators that need to be quantized include linear layers, normalization layers, and activation layers;

[0117] Integrate the parameter matrix and bias value corresponding to the linear layer into the table items of the first lookup table to obtain a third lookup table;

[0118] The normalization layer maps the feature vector before normalization to the corresponding table entry by applying an inverse linear transformation to the partition threshold in the third lookup table to obtain a fourth lookup table;

[0119] The activation layer calculates the inverse function of the activation function, and applies the calculation result of the inverse function to the partition threshold of the fourth lookup table to obtain the second lookup table.

[0120] Furthermore, the above-mentioned reasoning module 205 is specifically used for:

[0121] The programmable switch receives data packets from the network port, dynamically parses the data packets according to the protocol through the parser of the programmable data plane, and saves the parsed feature vectors required by the neural network model in the metadata in advance;

[0122] Based on the metadata, start the matching action table pipeline of the programmable switch;

[0123] The matching action table pipeline obtains metadata, matches the metadata with the second lookup table through the ternary content addressable memory, and performs data inference according to the matching action table pipeline based on the matching result to obtain an inference result.

[0124] Furthermore, the above-mentioned reasoning module 205 is also used for:

[0125] Matching the metadata with the partition threshold in the second lookup table through the ternary content addressable memory, indexing to the corresponding target table entry, and retrieving the pre-cached target result from the target table entry;

[0126] The target results are summed to obtain the output vector, and the output is stored in the metadata and transmitted to the matching action table of the next stage to execute the neural network operator of the next stage. The above steps are repeated until the matching action table pipeline is completed and the inference result is obtained.

[0127] Furthermore, the above device is also used for:

[0128] Store the inference result in a specified state register for subsequent inference; or

[0129] Process the data packet accordingly based on the inference result; or

[0130] The inference results are encapsulated into data packets through the reverse parser and output through the specified network port.

[0131] According to the fast inference deployment device for programmable switches proposed in an embodiment of the present invention, the programming capabilities of programmable switches are utilized in ultra-high-speed networks. While performing data packet exchange, a pre-trained neural network model is implemented through the line-speed flow table resources of the programmable data plane to complete the data packet inference task of the data plane, thereby saving computing resources and achieving high-speed inference tasks.

[0132] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0133] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0134] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A fast reasoning deployment method for programmable switches, characterized in that: The method comprises: Obtain a training dataset for training a neural network model; Based on the training data set, determining a first lookup table corresponding to the neural network model; Obtaining a neural network model calculation graph corresponding to the neural network model; Based on the neural network model calculation graph and the first lookup table, integrating operators that need to be quantized in the neural network into a second lookup table, and deploying the second lookup table in a programmable switch; The programmable switch receives a data packet, and performs data inference on the data packet to obtain an inference result; Wherein, determining the first lookup table corresponding to the neural network model based on the training data set includes: Splitting the features of the training data set into multiple feature subspaces; An initial binary decision tree is constructed by an unsupervised decision tree algorithm, wherein each tree node in the initial binary decision tree includes a threshold value and an index value; Taking the initial bucket corresponding to the training data on each feature subspace as the root node of the initial binary decision tree, and performing bucket division on the nodes in the initial binary decision tree until a target binary decision tree is constructed; Determine the mean of each index in the bucket at the leaf node of the target binary decision tree as the corresponding new cache centroid, and determine each new cache centroid as the first lookup table corresponding to the neural network model; The step of integrating the operators that need to be quantized of the neural network into the second lookup table based on the neural network model calculation graph and the first lookup table includes: Based on the neural network model calculation graph, an operator that needs to be quantized is obtained, wherein the operator that needs to be quantized includes a linear layer, a normalization layer, and an activation layer; Integrating the parameter matrix and bias value corresponding to the linear layer into the table entries of the first lookup table to obtain a third lookup table; The normalization layer maps the feature vector before normalization to the corresponding table entry by applying an inverse linear transformation to the division threshold in the third lookup table to obtain a fourth lookup table; The activation layer calculates an inverse function of the activation function, and applies the calculation result of the inverse function to the partition threshold of the fourth lookup table to obtain a second lookup table.

2. The method according to claim 1, characterized in that The initial bucket corresponding to the training data on each feature subspace is used as the root node of the initial binary decision tree, and the nodes in the initial binary decision tree are divided into buckets until a target binary decision tree is constructed, including: Taking the initial bucket corresponding to the training data on each feature subspace as the root node of the initial binary decision tree; On the node, traverse each index in the bucket and determine the best split point at each index; Obtaining a threshold and an index value of the node based on the target segmentation point, and determining a first sub-bucket divided into the bucket as a left node of the node and a second sub-bucket as a right node of the node; Repeat the above steps until the target binary decision tree is constructed.

3. The method according to claim 1, characterized in that The programmable switch receives a data packet, and performs data reasoning on the data packet to obtain a reasoning result, including: The programmable switch receives a data packet from a network port, dynamically parses the data packet according to a specific protocol through a parser of a programmable data plane, and pre-saves the parsed feature vector required by the neural network model in metadata; Based on the metadata, starting a matching action table pipeline of the programmable switch; The matching action table pipeline obtains the metadata, matches the metadata with the second lookup table through a ternary content addressable memory, and performs data inference according to the matching action table pipeline based on the matching result to obtain an inference result.

4. The method according to claim 3, characterized in that The step of matching the metadata with the second lookup table by using a ternary content addressable memory, and performing data inference according to the matching action table pipeline based on the matching result to obtain an inference result includes: Matching the metadata with the partition threshold in the second lookup table through a ternary content addressable memory, indexing to a corresponding target table entry, and retrieving a pre-cached target result from the target table entry; The target results are summed to obtain an output vector, and the output is stored in the metadata and transmitted to the matching action table of the next stage to execute the neural network operator of the next stage. The above steps are repeated until the matching action table pipeline is completed and the inference result is obtained.

5. The method according to claim 1, characterized in that The method further comprises: storing the inference result in a specified stateful register for subsequent inference; or Processing the data packet accordingly according to the inference result; or The inference result is encapsulated into the data packet through the reverse parser and output through the designated network port.

6. A fast reasoning deployment device for a programmable switch, characterized in that: The device comprises: A first acquisition module is used to acquire a training data set for training a neural network model; A determination module, configured to determine a first lookup table corresponding to the neural network model based on the training data set; A second acquisition module is used to acquire a neural network model calculation graph corresponding to the neural network model; A deployment module, configured to integrate operators that need to be quantized of the neural network into a second lookup table based on the neural network model calculation graph and the first lookup table, and deploy the second lookup table in a programmable switch; An inference module, used for the programmable switch to receive a data packet and perform data inference on the data packet to obtain an inference result; The determining module is specifically used for: Splitting the features of the training data set into multiple feature subspaces; An initial binary decision tree is constructed by an unsupervised decision tree algorithm, wherein each tree node in the initial binary decision tree includes a threshold value and an index value; Taking the initial bucket corresponding to the training data on each feature subspace as the root node of the initial binary decision tree, and performing bucket division on the nodes in the initial binary decision tree until a target binary decision tree is constructed; Determine the mean of each index in the bucket at the leaf node of the target binary decision tree as the corresponding new cache centroid, and determine each new cache centroid as the first lookup table corresponding to the neural network model; The deployment module is specifically used for: Based on the neural network model calculation graph, an operator that needs to be quantized is obtained, wherein the operator that needs to be quantized includes a linear layer, a normalization layer, and an activation layer; Integrating the parameter matrix and bias value corresponding to the linear layer into the table entries of the first lookup table to obtain a third lookup table; The normalization layer maps the feature vector before normalization to the corresponding table entry by applying an inverse linear transformation to the division threshold in the third lookup table to obtain a fourth lookup table; The activation layer calculates an inverse function of the activation function, and applies the calculation result of the inverse function to the partition threshold of the fourth lookup table to obtain a second lookup table.

7. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images