A multi-branch attention table model without decoder and a construction method and application thereof

Abstract

This invention discloses a decoder-free multi-branch attention table large-scale model and its reinforcement learning training method. Addressing the problems of redundant decoder structures, high computational overhead, and mismatch between training objectives and evaluation metrics in traditional supervised learning models, this invention proposes a multi-branch attention network based on an encoder-only architecture. This method removes the decoder portion from the traditional Transformer architecture, directly extracting feature interaction information using a multi-branch encoder. During the training phase, the Group Relative Policy Optimization (GRPO) algorithm is introduced for reinforcement learning training, abandoning the Critic network in traditional RL and directly calculating the relative advantage by sampling a set of outputs from the same input. This invention significantly reduces the number of model parameters and memory usage, while improving the accuracy and inference speed of table data processing by directly optimizing sequence-level rewards.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and machine learning technology, specifically to a deep neural network architecture for processing structured tabular data and its reinforcement learning-based optimization training method. Background Technology

[0002] With the development of deep learning, the Transformer architecture has been widely used in tabular data processing. In terms of model architecture, Amazon researchers proposed TabTransformer [1], which uses the Transformer encoder to map classification features to contextual embeddings. The SAINT network [2] proposed by the Kakao Brain team introduces a row and column attention mechanism, which further improves performance. Existing technologies have also proposed a model based on a multi-branch attention mechanism [3], which effectively captures the complex interactions between tabular features.

[0003] However, the aforementioned prior art has the following drawbacks:

[0004] 1. Redundancy in architecture: Reference [3] and some generative table models retain the decoder structure of Transformer. For discriminative tasks such as classification or regression, the presence of the decoder increases the number of unnecessary parameters and inference delay, and the autoregressive generation process is often not necessary in table prediction.

[0005] 2. Dependence on the full training data in the inference stage: In the architecture design of the reference [3], the decoder module is not only used for feature decoding, but is also designed as a mechanism similar to retrieval or non-parametric attention. Specifically, in the model inference stage, the decoder needs to feed the full training set data (or its feature representation) as input (Key / Value) into the network so that the test sample can interact with the training sample to perform attention calculation. This design leads to the inference computation and memory overhead increasing linearly or even superlinearly with the size of the training set. When the amount of training set data reaches hundreds of thousands or millions, the inference stage needs to consume a huge amount of memory to store the training set features, and each prediction needs to calculate the attention weights of the test sample and all training samples, resulting in extremely high inference latency, which cannot meet the needs of real-time response in industry.

[0006] 3. Mismatch between training objectives and metrics: Existing technologies often use cross-entropy or mean squared error as loss functions. These "surrogate losses" differ from the final business evaluation metrics (such as F1-Score, AUC, or custom business revenue), leading to inaccurate model optimization directions.

[0007] 4. Traditional reinforcement learning (RL) has high overhead: Although the introduction of RL can solve the problem of index mismatch, the mainstream PPO algorithm [4] usually requires an additional Critic network to estimate the advantage function, which makes the memory usage and training computation almost double. Recent studies such as DeepSeekMath [5] have proposed a GRPO algorithm that removes the Critic, but it has not yet been applied to the optimization of lightweight tabular data models. Summary of the Invention

[0008] To address the shortcomings and deficiencies in existing technologies, the present invention aims to provide a large multi-branch attention table model without a decoder and a reinforcement learning training method to solve problems such as redundant architecture, inference dependence on the entire training data, mismatch between training objectives and metrics, and high overhead of traditional reinforcement learning.

[0009] The proposed method for constructing a large multi-branch attention table model without a decoder includes the following steps:

[0010] Step 1: Decoder model construction;

[0011] Step 2: GRPO reinforcement training, wherein the GRPO reinforcement training employs a group relative policy optimization algorithm, including:

[0012] Step 21: Group sampling. For each input table sample x, the model performs sampling according to the current strategy. Sample to generate a set of outputs Where G is the group size;

[0013] Step 22: Reward Calculation. Calculate the reward value for each output within the group based on the preset business evaluation indicators. ;

[0014] Step 23: Critic-free advantage estimation. Calculate the advantage value of the i-th output using the mean and standard deviation of the rewards for this set of outputs. The calculation formula is: Where ϵ is the minimum value to prevent the denominator from being zero;

[0015] Step 24: Calculate the probability ratio and the current update strategy. With sampling strategy In output The probability ratio above:

[0016]

[0017] Step 25: Construct the truncated agent objective function, the policy objective of the i-th sample. The clip(v,min,max) function restricts the value v to the interval [min,max], and ε is a hyperparameter.

[0018] Step 26: Constructing the total loss function. Where D represents the distribution of the training dataset, and β is the KL divergence penalty coefficient. This is a strategy for reference.

[0019] In step 1 of this invention, the de-decoder model construction includes:

[0020] Step 11: Construction of the feature embedding layer, which receives the classification features and numerical features of the tabular data and maps them into feature vectors of a unified dimension through a lookup table and a linear layer, respectively;

[0021] Step 12: Decoder-free multi-branch encoder construction, including branch partitioning, multi-branch interaction, and structural simplification. Branch partitioning involves dividing the input feature vector sequence into multiple subsets, with each branch focusing on a different feature subspace. Multi-branch interaction involves applying a multi-head self-attention mechanism within each branch and integrating global information through an inter-branch fusion module. Structural simplification involves removing the decoder module after the encoder and its corresponding mask attention operation.

[0022] Step 13: Construction of the policy prediction head. The prediction head is a fully connected layer that is directly connected to the output of the encoder to output the action probability distribution.

[0023] In step 12 of this invention, the number of attention heads and the dimension of each head in the multi-head self-attention mechanism are determined through optimization during the training process, and the optimization results are different for different datasets and different tasks.

[0024] In step 12 of this invention, the fusion method of the inter-branch fusion module is weighted average, and the weights are obtained by the moving average during the training process.

[0025] In step 22 of this invention, the preset business evaluation indicators include at least one of prediction accuracy, business revenue value, F1-Score, and AUC.

[0026] In step 23 of this invention, the minimum value ϵ takes the value of .

[0027] In step 25 of this invention, the hyperparameter ε is 0.1 or 0.2.

[0028] In step 26 of this invention, the reference strategy To initialize the model or SFT model.

[0029] Based on the above methods, this invention also proposes a large multi-branch attention table model without a decoder, including:

[0030] The feature embedding layer receives the categorical and numerical features of tabular data and maps them into feature vectors of a uniform dimension through a lookup table and a linear layer, respectively.

[0031] The multi-branch encoder without a decoder includes multiple branches and an inter-branch fusion module. The multiple branches are used to partition the feature vector sequence into subsets and extract features through a multi-head self-attention mechanism. The inter-branch fusion module is used to integrate the global information of each branch. The multi-branch encoder removes the decoder module and its corresponding mask attention operation.

[0032] The policy prediction head is a fully connected layer connected to the output of the multi-branch encoder, used to output the action probability distribution.

[0033] In this invention, the number of branches of the multi-branch encoder and the feature subspace division of each branch are determined through the training process to adapt to different tabular data processing tasks.

[0034] This invention also proposes an application of a multi-branch attention table large model for decoders in classification tasks.

[0035] The classification described in this invention is considered to include binary classification tasks or quadruple classification tasks; the quadruple classification task includes testing the robustness of AutoML (Automatic Machine Learning) systems.

[0036] The large multi-branch attention table model without decoder disclosed in this invention has the following beneficial effects:

[0037] Model parameter reduction: Compared with reference [3], this invention removes the Decoder module. Since the Decoder usually contains self-attention and cross-attention layers, the number of model parameters is reduced by about 30%-50% after removal, and the serial bottleneck of autoregression generation is eliminated, and the inference latency is significantly reduced.

[0038] Completely decouples inference from training data, significantly reducing inference complexity: The decoder in reference [3] requires the input of the entire training set during inference, and its inference time complexity and space complexity are... This makes the model almost unusable in big data scenarios. This invention removes the decoder and reconstructs the multi-branch encoder, so that the model no longer needs to be "fed" any training set data during the inference phase. With the model parameters fixed, the computational cost for inference on a single sample is constant O(1) (dependent only on the number of model layers, not on the size of the training set). This improvement significantly reduces the model's memory usage and increases inference speed.

[0039] Memory efficiency optimization of reinforcement learning algorithm: Compared with the standard PPO algorithm [4], the present invention adopts the GRPO algorithm [5], which does not require loading and training a Critic network of the same size as the Policy network, and supports a larger batch size or a more complex encoder structure under the same hardware.

[0040] Improved performance and accuracy: Compared with supervised learning methods such as TabTransformer [1], this invention directly optimizes the non-differentiable final index (such as AUC) through RL, and combined with the feature extraction capability of multi-branch structure, it can achieve higher accuracy on table tasks. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram showing the overall structure of the decoder-free multi-branch attention table Transformer provided in this embodiment of the invention;

[0043] Figure 2 The flowchart illustrates the reinforcement learning training logic based on GRPO provided in this embodiment of the invention, demonstrating the advantages of computation without the need for a Critic network. Detailed Implementation

[0044] The invention will be further described in detail below with reference to the specific embodiments and accompanying drawings. Except for the contents specifically mentioned below, the processes, conditions, and experimental methods for implementing the invention are all common knowledge and general knowledge in the art, and the invention does not have any particular limitations.

[0045] This invention discloses a decoder-free multi-branch attention table large-scale model and its reinforcement learning training method. Addressing the problems of redundant decoder structures, high computational overhead, and mismatch between training objectives and evaluation metrics in traditional supervised learning models, this invention proposes a multi-branch attention network based on an encoder-only architecture. This method removes the decoder part from the traditional Transformer architecture, directly extracting feature interaction information using a multi-branch encoder. During the training phase, the Group Relative Policy Optimization (GRPO) algorithm is introduced for reinforcement learning training, abandoning the Critic network in traditional RL and directly calculating the relative advantage by sampling a set of outputs from the same input. This invention significantly reduces the number of model parameters and memory usage, while improving the accuracy and inference speed of table data processing by directly optimizing sequence-level rewards.

[0046] This invention proposes a method for constructing a large multi-branch attention table model without a decoder, comprising the following steps:

[0047] Step 1: Decoder model construction;

[0048] Step 2: GRPO reinforcement training.

[0049] Specifically, the de-decoder model construction includes the following sub-steps:

[0050] Step 11, Feature Embedding Layer: Receives the categorical and numerical features of the tabular data, and maps them into feature vectors of a unified dimension through a lookup table and a linear layer, respectively.

[0051] Step 12, De-decode the multi-branch encoder:

[0052] Branching: The input feature vector sequence is divided into multiple subsets (Branches), with each branch focusing on a different feature subspace;

[0053] Multi-branch interaction: A multi-head self-attention mechanism (MHA) is applied within each branch, and global information is integrated through an inter-branch fusion module. Hyperparameters such as the number of attention heads and the dimension of each head are tuned during training, and the tuning results vary for different datasets and tasks. The fusion method is a weighted average, and the weights are obtained during training using a moving average. Specific details are provided in section 3.4.2 and Figure 3a of reference 3 (Multi-branch of Attention Yields Accurate Results for Tabular Data).

[0054] Simplified structure: The decoder module and its corresponding mask attention operation located after the encoder in the reference [3] are directly removed.

[0055] Policy prediction head: The prediction head is a simple lookahead layer, see Equation 5 in the reference. A fully connected layer is directly connected to the encoder output (such as the global average pooling vector) to output the action probability distribution. .

[0056] Specifically, the GRPO reinforcement training uses Group Relative Policy Optimization (GRPO) instead of traditional supervised learning or PPO-Critic architecture, and includes the following sub-steps:

[0057] Step 21, Group Sampling: For each input table sample x, the model performs sampling according to the current policy. Sample to generate a set of outputs Where G is the group size, 5-8;

[0058] Step 22, Reward Calculation: Calculate the reward value for each output within the group based on preset business evaluation indicators (such as prediction accuracy and business revenue). ;

[0059] Step 23, Critic-free advantage estimation: Instead of using a value network, calculate the advantage value of the i-th output using the mean and standard deviation of the rewards from the group's outputs. :

[0060] ;

[0061] in, This represents the original reward value corresponding to the i-th output. This represents the average of the G rewards within a group; This represents the standard deviation of the G rewards within a group; To prevent the minimum value where the denominator is zero (e.g.) ).

[0062] Step 24, Probability Ratio Calculation: Calculate the current update strategy With sampling strategy In output The probability ratio :

[0063] ;

[0064] Step 25, Truncate the agent objective function: To prevent excessively large policy update steps from causing training instability, a PPO-style truncation mechanism is adopted. For the policy objective function of the i-th sample... Defined as:

[0065]

[0066] in, The function restricts the value v to the interval [min, max]. This is a hyperparameter (usually 0.1 or 0.2) used to limit the range of variation of the probability ratio; The advantage estimate for the i-th output sample is calculated in step 23.

[0067] Step 26, Total Loss Function: To prevent the current policy Deviation from reference strategy (Typically, this is due to the initialization model or SFT model) being too far away, so KL divergence is introduced as a regularization term. Finally, the total loss function... Defined as (with minimization as the optimization objective):

[0068]

[0069] Where D represents the distribution of the training dataset; The KL divergence penalty coefficient is used to control the strength of regularization; the negative sign before the symbol indicates that "maximizing reward" is converted to "minimizing loss". .

[0070] By using the loss function described above, the gradient direction can be estimated by utilizing the relative merits of the same set of sampling results without maintaining the Critic network, thereby efficiently updating the model weights.

[0071] Example 1: Binary Classification Task Based on Financial Risk Control Data

[0072] Data preprocessing: Select a dataset that includes user age, income (numerical) and occupation, residence (categorical).

[0073] Model building: Constructing a multi-branch encoder and setting feature dimensions Number of branches .

[0074] Only retain the Encoder layer (number of layers) Remove the Decoder layer. The prediction header outputs the probabilities of "default" and "non-default".

[0075] GRPO Training: Setting Group Sampling Size For each sample, the model samples the prediction results 8 times.

[0076] Define a reward function: if the predicted result matches the true label, the reward is... ;otherwise .

[0077] Calculate the average reward for these 8 results. and standard deviation .

[0078] Calculate the advantage of each result .

[0079] The model weights are updated using a PPO-clip style loss function without needing to update any Critic network.

[0080] Experimental conclusions: Compared to the previous version with the Decoder (Hidden_size=128, encoder_layers_num=16, decoder_layer_num=5), the inference speed of the model in this embodiment is around 260ms, which is an improvement of 40%; compared to the version trained using cross-entropy, the AUC index is improved by 1.2%.

[0081] Example 2: Binary Classification Task for Tabular Data

[0082] The Adult dataset is a standard public dataset in the field of tabular data. It is a binary classification dataset with 14 features (6 numerical and 8 categorical).

[0083] Objective: To predict whether a person's annual income exceeds $50,000.

[0084] Model parameters: 14 encoder layers, d=64 feature dimension, and 7 branches.

[0085] Experimental conclusion: Compared with the previous version (decoder_layer_num=6), the implementation model has an iteration speed of about 130ms with batch_size=1024, which is a 40% improvement, while the accuracy remains unchanged (0.8633 vs. 0.8632).

[0086] Example 3: Testing the robustness of the AutoML (Automated Machine Learning) system

[0087] The Jannis dataset is a standard public dataset in the field of tabular data. The Jannis dataset is a four-class classification dataset with 54 features as input, used to test the robustness of AutoML (Automated Machine Learning) systems.

[0088] Model parameters: 1 encoder layer, d=128 feature dimension, 4 branches.

[0089] Experimental conclusion: Compared with the previous version (decoder_layer_num=6), the implementation model has an iteration speed of about 855ms with batch_size=128, which is 40% faster, and the accuracy is also improved (0.7286 vs. 0.7207).

[0090] References:

[0091] [1] Xin Huang, Ashish Khetan, Milan Cvitkovic, Zohar Karnin. "TabTransformer: Tabular Data Modeling Using Contextual Embeddings". arXivpreprint arXiv:2012.06678, 2020.

[0092] [2] Gowthami Somepalli, et al. "SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training". arXiv preprintarXiv:2106.01342, 2021.

[0093] [3] Li, Xuechen; Li, Yupeng; Liu, Jian; Jin, Xiaolin; Hu, Xin.“Multi-branch of Attention Yields Accurate Results for Tabular Data.” arXivpreprint arXiv:2502.12507, 2025.

[0094] [4] John Schulman, et al. "Proximal Policy Optimization Algorithms".arXiv preprint arXiv:1707.06347, 2017.

[0095] [5] Zhihong Shao, et al. "DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models". arXiv preprint arXiv:2402.03300, 2024.

[0096] Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0097] As used in this invention, the term "comprising" is an open-ended expression, meaning it includes the contents specified in this invention but does not exclude other aspects.

[0098] As used in this invention, the term "and / or" includes any one or more of the related listed items and all combinations thereof.

[0099] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of the inventive concept are included in this invention and are protected by the appended claims.

Claims

1. A method for constructing a large multi-branch attention table model without a decoder, characterized in that, Includes the following steps: Step 1: Decoder model construction; Step 2: GRPO reinforcement training, wherein the GRPO reinforcement training employs a group relative policy optimization algorithm, including: Step 21: Group sampling. For each input table sample x, the model performs sampling according to the current strategy. Sample to generate a set of outputs Where G is the group size; Step 22: Reward Calculation. Calculate the reward value for each output within the group based on the preset business evaluation indicators. ; Step 23: Critic-free advantage estimation. Calculate the advantage value of the i-th output using the mean and standard deviation of the rewards for this set of outputs. The calculation formula is: ;in, This represents the original reward value corresponding to the i-th output. This represents the average of the G rewards within a group; This represents the standard deviation of the G rewards within the calculation group; ϵ is a minimum value to prevent the denominator from being zero. Step 24: Calculate the probability ratio and the current update strategy. With sampling strategy In output The probability ratio above: Step 25: Construct the truncated agent objective function, the policy objective of the i-th sample. The clip(v,min,max) function restricts the value v to the interval [min,max], and ε is a hyperparameter. The advantage estimate for the i-th output sample is calculated in step 23; Step 26: Constructing the total loss function. Where D represents the distribution of the training dataset, and β is the KL divergence penalty coefficient. For reference strategy; .

2. The construction method according to claim 1, characterized in that, In step 1, the de-decoder model construction includes: Step 11: Construction of the feature embedding layer, which receives the classification features and numerical features of the tabular data and maps them into feature vectors of a unified dimension through a lookup table and a linear layer, respectively; Step 12: Decoder-free multi-branch encoder construction, including branch partitioning, multi-branch interaction, and structural simplification. Branch partitioning involves dividing the input feature vector sequence into multiple subsets, with each branch focusing on a different feature subspace. Multi-branch interaction involves applying a multi-head self-attention mechanism within each branch and integrating global information through an inter-branch fusion module. Structural simplification involves removing the decoder module after the encoder and its corresponding mask attention operation. Step 13: Construction of the policy prediction head. The prediction head is a fully connected layer that is directly connected to the output of the encoder to output the action probability distribution.

3. The construction method according to claim 2, characterized in that, In step 12, the number of attention heads and the dimension of each head in the multi-head self-attention mechanism are determined through tuning during the training process, and the tuning results are different for different datasets and different tasks. And / or, In step 12, the fusion method of the inter-branch fusion module is weighted average, and the weights are obtained by the moving average during the training process.

4. The construction method according to claim 1, characterized in that, In step 22, the preset business evaluation indicators include at least one of prediction accuracy, business revenue value, F1-Score, and AUC.

5. The construction method according to claim 1, characterized in that, In step 23, the minimum value ϵ takes the value of .

6. The construction method according to claim 1, characterized in that, In step 25, the hyperparameter ε is set to 0.1 or 0.

2.

7. The construction method according to claim 1, characterized in that, In step 26, refer to the strategy. To initialize the model or SFT model.

8. A large multi-branch attention table model without decoder, characterized in that, include: The feature embedding layer receives the categorical and numerical features of tabular data and maps them into feature vectors of a uniform dimension through a lookup table and a linear layer, respectively. The multi-branch encoder without a decoder includes multiple branches and an inter-branch fusion module. The multiple branches are used to partition the feature vector sequence into subsets and extract features through a multi-head self-attention mechanism. The inter-branch fusion module is used to integrate the global information of each branch. The multi-branch encoder removes the decoder module and its corresponding mask attention operation. The policy prediction head is a fully connected layer connected to the output of the multi-branch encoder, used to output the action probability distribution.

9. The large table model according to claim 8, characterized in that, The number of branches and the feature subspace division of each branch in a multi-branch encoder are determined through the training process to adapt to different tabular data processing tasks.

10. The application of the large table model as described in claim 7 or 8 in classification tasks.

Images