Decision mixer construction method, device and equipment for dynamic token selection

Abstract

The invention provides a decision mixer construction method, device and equipment for dynamic token selection, and the method comprises the steps: constructing a model framework of a decision mixer; the decision mixer comprises a mixing layer and a standard Transform layer which are alternately stacked, and the mixing layer comprises a dynamic token selection mechanism; performing end-to-end training on the decision mixer, and synchronously learning dichotomy probability distribution of token selection results based on the dynamic token selection mechanism through an auxiliary predictor; and dynamically optimizing and adjusting a dynamic token selection mechanism of the mixing layer based on an output result of the auxiliary predictor, and determining a final model framework of the decision mixer. According to the invention, the adaptive balance of the long-term dependence feature and the local Markov feature is realized, and the problem that the features are difficult to effectively balance in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of offline reinforcement learning, and in particular to a method, device and equipment for constructing a decision mixer for dynamic token selection. Background Art

[0002] In offline reinforcement learning, conditional sequence modeling methods generate future action decisions by modeling the sequence of states, actions, and rewards in historical trajectories. Transformer-based decision transformers and their variants have become mainstream architectures, capturing sequence dependencies through global attention or local convolution.

[0003] However, the existing technology has the following limitations: feature balance problem: the decision transformer over-relies on the global attention mechanism and has difficulty in effectively capturing the local Markov features of adjacent time steps; the decision convolver over-emphasizes local correlations through fixed-range convolution, resulting in insufficient long-term dependency modeling capabilities, especially in non-standard Markov environments (such as state-dependent multi-step history), where performance is significantly degraded; low computational efficiency: the decision transformer directly expands the model size to improve performance, but the computing resource consumption increases quadratically with the sequence length, making it difficult to adapt to large-scale tasks; autoregressive sampling myopia: existing methods rely on complete sequence information when generating actions, resulting in decision myopia in the inference stage due to the inability to predict future tokens, affecting the stability of the strategy; suboptimal trajectory interference: the suboptimal trajectory fragments contained in the training data will interfere with the model's extraction of key features. The existing architecture lacks a dynamic filtering mechanism, which makes the model susceptible to noise.

[0004] There is currently no effective solution to the problem of difficulty in effectively balancing features in existing related technologies. Summary of the Invention

[0005] The present invention provides a decision mixer construction method, device and equipment for dynamic token selection, which are used to solve the defect of the related technology in the prior art that it is difficult to effectively balance features, significantly improve model performance and reduce the computational burden.

[0006] In a first aspect, the present invention provides a method for constructing a decision mixer for dynamic token selection, comprising: Constructing a model framework for a decision mixer; the decision mixer consists of alternating stacked mixing layers and standard Transformer layers, with the mixing layers including a dynamic token selection mechanism; Performing end-to-end training on the decision mixer and synchronously learning a binary probability distribution of token selection results based on the dynamic token selection mechanism through an auxiliary predictor; The dynamic token selection mechanism of the hybrid layer is dynamically optimized and adjusted based on the output result of the auxiliary predictor to determine the final model framework of the decision mixer.

[0007] According to a method for constructing a decision mixer for dynamic token selection provided by the present invention, the dynamic token selection mechanism is as follows: for each token in an input sequence, a corresponding weight value is generated by a router, and a token selection threshold is dynamically generated in combination with a super network; Based on the weight value of the token and the token selection threshold, the key token is screened out from the input sequence, and the key token is input into the attention calculation module for feature extraction; The output result of the attention calculation module is concatenated with the unfiltered tokens in the input sequence to generate a new input sequence.

[0008] According to a method for constructing a decision mixer for dynamic token selection provided by the present invention, training the auxiliary predictor includes: Recording each key token screened by the hybrid layer in real time and generating a corresponding binary classification label; the binary classification label is used to indicate whether the token is selected; Taking the embedding representation of the token as input, the auxiliary predictor is subjected to parameter optimization.

[0009] According to a method for constructing a decision mixer for dynamic token selection provided by the present invention, the method takes the embedding representation of the token as input and optimizes the parameters of the auxiliary predictor, comprising: Inputting the current embedding representation of the token into the auxiliary predictor and outputting the probability value of the token being selected; Based on the output result of the auxiliary predictor, the parameters of the auxiliary predictor are optimized by a cross entropy loss function.

[0010] According to a method for constructing a decision mixer for dynamic token selection provided by the present invention, the decision mixer comprises 2-4 layer groups, each layer group comprises at least one mixing layer and one standard Transformer layer; The standard Transformer layer is used to perform global attention calculation on the input sequence, stabilize the training process of the decision mixer and preserve global context information.

[0011] According to a method for constructing a decision mixer for dynamic token selection provided by the present invention, the computational efficiency of the dynamic token selection mechanism is optimized, including: The length of the input sequence input to the attention calculation module is reduced to 30%-70% of the original length through a screening mechanism; The original embedding representations of the tokens that are not filtered are retained in the residual connection.

[0012] According to a method for constructing a decision mixer for dynamic token selection provided by the present invention, the dynamic token selection mechanism of the hybrid layer is dynamically optimized and adjusted based on the output result of the auxiliary predictor, including: Predicting a probability value of the token being selected based on the embedding representation of the token at the current moment; The probability value of the token is compared with a preset threshold. If the probability value exceeds the preset threshold, the current mixed layer is activated to perform attention calculation, otherwise the token is directly passed.

[0013] According to a method for constructing a decision mixer for dynamic token selection provided by the present invention, end-to-end training of the decision mixer is performed, comprising: Obtaining a policy network and a history track of the decision mixer; the history track includes a state track, an action track, and a reward track; A loss function of the decision mixer is determined based on the policy network and the historical trajectory, and the decision mixer is trained with the goal of minimizing a function value of the loss function.

[0014] In a second aspect, the present invention further provides a decision mixer construction device for dynamic token selection, comprising: A building block for constructing a model framework for a decision mixer, wherein the decision mixer comprises alternating stacked mixing layers and standard Transformer layers, wherein the mixing layers include a dynamic token selection mechanism; A training module, configured to perform end-to-end training on the decision mixer and synchronously learn a binary probability distribution of token selection results based on the dynamic token selection mechanism through an auxiliary predictor; The inference module is used to dynamically optimize and adjust the dynamic token selection mechanism of the hybrid layer based on the output result of the auxiliary predictor to determine the final model framework of the decision mixer.

[0015] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the decision mixer construction method for dynamic token selection as described in the first aspect above is implemented.

[0016] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the decision mixer construction method for dynamic token selection as described in the first aspect above.

[0017] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the decision mixer construction method for dynamic token selection as described in the first aspect above.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method for constructing a decision mixer for dynamic token selection. During the training phase, the decision mixer is trained end-to-end based on offline reinforcement learning historical trajectory data. An auxiliary predictor is used to simultaneously learn the binary probability distribution of token selection results to compensate for the short-sightedness inherent in the autoregressive sampling process. During the inference phase, the dynamic token selection mechanism of each hybrid layer is dynamically adjusted based on the output of the auxiliary predictor to generate the final action prediction result. This achieves an adaptive balance between long-term dependency features and local Markov features, addressing the difficulty in effectively balancing features in existing related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 is a flow chart of a method for constructing a decision mixer for dynamic token selection provided by the present invention; Figure 2 Schematic diagram of the architecture of the decision mixer constructed in an embodiment of the present invention; Figure 3 It is a structural block diagram of a decision mixer construction device for dynamic token selection provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0022] The present invention provides a decision mixer construction method for dynamic token selection. Figure 1 Flowchart of the decision mixer construction method for dynamic token selection provided by the present invention, such as Figure 1 As shown, the method includes the following steps: Step S101, constructing a model framework of a decision mixer; the decision mixer includes alternately stacked mixing layers and standard Transformer layers, and the mixing layers include a dynamic token selection mechanism; Step S102 , performing end-to-end training on the decision mixer, and synchronously learning the binary classification probability distribution of token selection results based on the dynamic token selection mechanism through the auxiliary predictor; Step S103 : dynamically optimizing and adjusting the dynamic token selection mechanism of the hybrid layer based on the output result of the auxiliary predictor to determine the final model framework of the decision mixer.

[0023] In this method, a decision mixer model framework is first constructed. During the training phase, the decision mixer is trained end-to-end based on offline reinforcement learning historical trajectory data. An auxiliary predictor is used to simultaneously learn the binary probability distribution of token selection results to compensate for the short-sightedness of the autoregressive sampling process. During the inference phase, the dynamic token selection mechanism of each mixing layer is dynamically adjusted based on the output of the auxiliary predictor to generate the final action prediction result. This achieves an adaptive balance between long-term dependency features and local Markov features, solving the problem of effective feature balancing in existing related technologies.

[0024] In some of the embodiments, the dynamic token selection mechanism is as follows: for each token in the input sequence, a corresponding weight value is generated by a router, and a token selection threshold is dynamically generated in combination with a supernetwork; based on the weight value of the token and the token selection threshold, key tokens are screened out from the input sequence, and the key tokens are input into an attention calculation module for feature extraction; the output result of the attention calculation module is spliced ​​and fused with the unscreened tokens in the input sequence to generate a new input sequence.

[0025] In this embodiment, the expert routing mechanism in the Mixture of Experts (MOE) is used to refer to each token in the input sequence. The weight is generated by the router and the number of key tokens to be selected in the current layer is dynamically determined by combining the hypernetwork. The specific implementation is as follows: Let the current layer input sequence be , the router is , the super network is , calculate the weight value of each token, and determine the token selection threshold, sort the tokens according to the weight value, and select the top Through the synergy of routers and hypernetworks, the threshold can be adaptively determined according to the data characteristics, so that the front is selected based on the weight output by the router. tokens participate in the attention calculation, while skipping the tokens that are not selected in the current layer.

[0026] The selected key tokens are input into the attention calculation module for global feature extraction, and the unselected tokens are directly passed through the residual connection. Selected token After passing through the attention layer and the feedforward layer, the processed features are obtained , and The weight values ​​corresponding to the respective tokens in the multiplication are then concatenated with the unselected tokens. The position of each token after concatenation is the same as that of these tokens in The original position is exactly the same, the specific formula is as follows:

[0027] in, Indicates the Layer i tokens, Indicates the l Layer i The weight of the tokens, Represents the features after the attention layer and feedforward network, the shape is the same as Consistent, I Denotes an indicator function that outputs 1 if the value in the brackets is true and 0 otherwise. Standard Transformer layers are alternately used with mixing layers to ensure the stability of the decision mixer. Router weights are placed on the gradient propagation path, allowing them to receive feedback updates via gradient descent during training. This token selection mechanism enables the decision mixer to focus on exploring relationships between important tokens, avoiding interference from irrelevant or suboptimal tokens. Residual connections also ensure the continuous transmission of critical information.

[0028] For example, the offline reinforcement learning dataset is defined as , where the trajectory Contains the future reward, state, and action triples. Drawing on the expert routing mechanism in the hybrid expert architecture, the model framework is constructed. Each layer consists of a hybrid layer (dynamic selection module) alternating with a standard Transformer layer. Each layer initializes the router independently (fully connected networks) and hypernetworks (Multilayer Perceptron (MLP) outputs integer ), used to dynamically select the tokens of the current layer to participate in the attention calculation. Initialize a lightweight binary classifier for each layer (2-layer MLP) for token selection decisions in the inference phase.

[0029] Track The intercept length is The context window of , the specific formula is as follows:

[0030] in, represents a future reward token, Represents a state token, represents an action token, and the embedding is represented as . l Layer Input , calculate the weight value of each token , and generate thresholds through the hypernetwork = . Select the top one with the highest weight tokens constitute a subsequence , retaining the original timing order. Input attention calculation module calculates features , is concatenated with the unselected tokens after residual connection. The specific formula is as follows:

[0031] in, represents the weight value of the token, Represents token characteristics, Indicates the Layer i tokens.

[0032] In some embodiments, training the auxiliary predictor includes: recording the key tokens screened by each mixing layer in real time and generating corresponding binary classification labels; the binary classification labels are used to characterize whether the tokens are selected; and optimizing the parameters of the auxiliary predictor using the embedded representation of the tokens as input.

[0033] Specifically, the auxiliary predictor is optimized with the embedding representation of the token as input, including: inputting the embedding representation of the current token into the auxiliary predictor and outputting the probability value of the token being selected; based on the output result of the auxiliary predictor, the auxiliary predictor is optimized through the cross entropy loss function.

[0034] Based on the above embodiment, the decision mixer is trained end-to-end, including: obtaining the policy network and historical trajectory of the decision mixer; the historical trajectory includes the state trajectory, action trajectory and reward trajectory; determining the loss function of the decision mixer based on the policy network and the historical trajectory, and training the decision mixer with the goal of minimizing the function value of the loss function.

[0035] The dynamic token selection mechanism achieves adaptive adjustment of feature balance while significantly reducing computational costs. The generation of depends on the information of the complete sequence, which means that the selection of a specific token is influenced by both historical tokens and future tokens. Although this mechanism is straightforward to implement during the training phase, it is hindered during the autoregressive inference process of gradually generating tokens. To address this issue, this method introduces an auxiliary predictor , this module receives a single token input and directly predicts whether the current token should participate in the calculation of the attention mechanism layer at the token level. That is, input the current token , output binary probability , and trained using cross entropy loss.

[0036] The operating logic of the auxiliary predictor is shown in the following formula:

[0037] in, is the output of binary classification, =1 means selected, =0 means not selected, is the indicator function, is the sigmoid activation function, The input sequence is Layer The residual connection retains the information of unselected tokens and ensures the stability of long sequence generation.

[0038] In some of the embodiments, the computational efficiency of the dynamic token selection mechanism is optimized, including: reducing the input sequence length of the input attention calculation module to 30%-70% of the original length through a screening mechanism; and retaining the original embedding representation of the unscreened tokens in the residual connection.

[0039] For example, for action prediction loss: set the action prediction loss to minimize the mean squared error between the model output action and the true action:

[0040] in, represents the mean squared error of actions for training the decision mixer, Represents the decision mixer at time step Predicted actions, Represents the time step The real action label is used to optimize the decision mixer through the action prediction loss, and the auxiliary predictor is optimized synchronously through cross entropy. The specific formula is as follows:

[0041] in, represents the cross entropy error for training the auxiliary predictor, represents the length of the current sequence, and S =3 K , i Indicates the token number, The value is 0 or 1, indicating whether the current token is selected. 1 means it is selected. 0 means not selected. represents the logarithmic probability of the auxiliary predictor predicting that the i-th token will be selected, Represents the sigmoid activation function. During training, the decision mixer and the auxiliary predictor are trained synchronously, but the training process is independent of each other. That is, the auxiliary predictor is not part of the decision mixer during training. During inference, the auxiliary predictor replaces the hypernetwork structure in the decision mixer to select tokens.

[0042] The value of k is dynamically adjusted based on task characteristics to achieve a balance between long-term and short-term dependencies. For standard Markov tasks (such as HalfCheetah), the value of k approaches the local window, degenerating into a local convolution pattern similar to the decision convolutional layer. For non-standard Markov tasks (such as Maze2D), the value of k is increased to capture long-range dependencies, approaching the global attention mechanism of the original decision transformer. From a computational complexity perspective, if the length of the sequence entering the attention computation module is halved, the floating point operations per second (FLOPs) consumption of the key-value matrix multiplication is reduced to 25% of the original value. Similar calculations can be used to calculate the FLOPs savings of the MLP module.

[0043] For the overall training process, the token selection results of each hybrid layer are recorded in real time during the training process to generate a binary classification label indicating whether it is selected; the embedding representation of the current token is input into the auxiliary predictor, and the probability value of its selection is output; through the cross entropy loss function Optimize the predictor parameters to achieve online learning of the token selection strategy. The specific formula is as follows:

[0044] in, represents the cross entropy error for training the auxiliary predictor, represents the length of the current sequence, and S =3 K , i Indicates the token number, The value is 0 or 1, indicating whether the current token is selected. 1 means it is selected. 0 means not selected. represents the logarithmic probability of the auxiliary predictor predicting that the i-th token will be selected, represents a sigmoid activation function. In some embodiments, the decision mixer includes 2-4 layer groups, each of which includes at least one mixing layer and one standard Transformer layer; the standard Transformer layer is used to perform global attention calculations on the input sequence, stabilize the training process of the decision mixer, and preserve global context information.

[0045] In some of the embodiments, the dynamic token selection mechanism of the hybrid layer is dynamically optimized and adjusted based on the output results of the auxiliary predictor, including: predicting the probability value of the token being selected based on the embedded representation of the token at the current moment; comparing the probability value of the token with a preset threshold, if the probability value exceeds the preset threshold, activating the current hybrid layer to perform attention calculation, otherwise directly passing the token.

[0046] In summary, if Figure 2 As shown, Figure 2 This is a schematic diagram of the architecture of the decision mixer constructed in the embodiment of the present invention. In the training phase, the input sequence Will be based on The layer mixer's routers and hypernetwork perform dynamic token selection; during the inference sampling phase, the decision mixer uses an auxiliary predictor to filter key tokens from the input sequence. Each mixing layer independently trains its own dedicated router, hypernetwork, and auxiliary predictor parameters.

[0047] During the model training phase, a dynamic token selection mechanism is used to adaptively balance the modeling capabilities of long-term temporal dependency features and local Markov features, significantly reducing the complexity of attention calculations. The decision mixer is trained based on offline trajectory data, and during the training process, dynamic routing and hypernetwork are used to collaboratively optimize the selection strategy of key tokens, enabling the decision mixer to autonomously adjust the weights of local and global features according to different task characteristics while retaining the general representation capability for multi-task scenarios (i.e., cross-task generalization). During the inference phase, an auxiliary predictor is used to determine in real time the necessity of a token to participate in attention calculations, and residual connections are combined to retain the information transmission of unselected tokens, ensuring the stability of the autoregressive generation process. Finally, through dynamic pruning and feature splicing mechanisms, the ability to extract high-quality subsequences from suboptimal trajectories is achieved, thereby demonstrating excellent generalization performance in standard / non-standard Markov tasks, zero-shot scenarios, and multi-task environments, while significantly reducing model computational overhead.

[0048] The present invention also provides a decision mixer construction device for dynamic token selection. The decision mixer construction device for dynamic token selection provided by the present invention is described below. The decision mixer construction device for dynamic token selection described below and the decision mixer construction method for dynamic token selection described above can be referenced to each other. Figure 3 This is a structural block diagram of a decision mixer construction device for dynamic token selection provided by the present invention. Figure 3 As shown, the device includes: Building module 301, for building a model framework of a decision mixer; the decision mixer includes alternately stacked mixing layers and standard Transformer layers, and the mixing layers include a dynamic token selection mechanism; A training module 302 is configured to perform end-to-end training on the decision mixer and simultaneously learn the binary probability distribution of token selection results based on the dynamic token selection mechanism through an auxiliary predictor; The reasoning module 303 is used to dynamically optimize and adjust the dynamic token selection mechanism of the hybrid layer based on the output results of the auxiliary predictor to determine the final model framework of the decision mixer.

[0049] When the device is in use, construction module 301 first constructs the model framework of the decision mixer. During the training phase, training module 302 performs end-to-end training on the decision mixer based on offline reinforcement learning historical trajectory data. It also synchronously learns the binary probability distribution of token selection results through an auxiliary predictor to compensate for the short-sightedness problem in the autoregressive sampling process. During the inference phase, inference module 303 dynamically adjusts the dynamic token selection mechanism of each mixing layer based on the output of the auxiliary predictor to generate the final action prediction result, achieving an adaptive balance between long-term dependent features and local Markov features, solving the problem of difficult to effectively balance features in existing related technologies.

[0050] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4 As shown, the electronic device may include: a processor 401, a communications interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communications interface 402, and the memory 403 communicate with each other via the communication bus 404. The processor 401 may call logic instructions in the memory 403 to execute a decision mixer construction method for dynamic token selection, which includes: Building blocks for constructing a model framework for a decision mixer; the decision mixer consists of alternating stacks of mixing layers and standard Transformer layers, with a dynamic token selection mechanism included in the mixing layers; A training module for end-to-end training of the decision mixer and synchronous learning of the binary probability distribution of token selection results based on the dynamic token selection mechanism through an auxiliary predictor; The inference module is used to dynamically optimize and adjust the dynamic token selection mechanism of the hybrid layer based on the output results of the auxiliary predictor to determine the final model framework of the decision mixer.

[0051] Furthermore, the logic instructions in the aforementioned memory 403 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0052] In another aspect, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the decision mixer construction method for dynamic token selection provided by the above methods, which includes: Building blocks for constructing a model framework for a decision mixer; the decision mixer consists of alternating stacks of mixing layers and standard Transformer layers, with a dynamic token selection mechanism included in the mixing layers; A training module for end-to-end training of the decision mixer and synchronous learning of the binary probability distribution of token selection results based on the dynamic token selection mechanism through an auxiliary predictor; The inference module is used to dynamically optimize and adjust the dynamic token selection mechanism of the hybrid layer based on the output results of the auxiliary predictor to determine the final model framework of the decision mixer.

[0053] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the decision mixer construction method for dynamic token selection provided by the above methods, the method comprising: Building blocks for constructing a model framework for a decision mixer; the decision mixer consists of alternating stacks of mixing layers and standard Transformer layers, with a dynamic token selection mechanism included in the mixing layers; A training module for end-to-end training of the decision mixer and synchronous learning of the binary probability distribution of token selection results based on the dynamic token selection mechanism through an auxiliary predictor; The inference module is used to dynamically optimize and adjust the dynamic token selection mechanism of the hybrid layer based on the output results of the auxiliary predictor to determine the final model framework of the decision mixer.

[0054] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0055] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A decision mixer construction method for dynamic token selection, characterized in that: include: Constructing a model framework for a decision mixer; the decision mixer consists of alternating stacked mixing layers and standard Transformer layers, with the mixing layers including a dynamic token selection mechanism; Performing end-to-end training on the decision mixer and synchronously learning a binary probability distribution of token selection results based on the dynamic token selection mechanism through an auxiliary predictor; The dynamic token selection mechanism of the hybrid layer is dynamically optimized and adjusted based on the output result of the auxiliary predictor to determine the final model framework of the decision mixer.

2. The decision mixer construction method for dynamic token selection according to claim 1, characterized in that The dynamic token selection mechanism is as follows: for each token in the input sequence, a corresponding weight value is generated by the router, and a token selection threshold is dynamically generated in combination with the hypernetwork; Based on the weight value of the token and the token selection threshold, the key token is screened out from the input sequence, and the key token is input into the attention calculation module for feature extraction; The output result of the attention calculation module is concatenated with the unfiltered tokens in the input sequence to generate a new input sequence.

3. The decision mixer construction method for dynamic token selection according to claim 1, characterized in that: Training the auxiliary predictor comprises: Recording each key token screened by the hybrid layer in real time and generating a corresponding binary classification label; the binary classification label is used to indicate whether the token is selected; Taking the embedding representation of the token as input, the auxiliary predictor is subjected to parameter optimization.

4. The decision mixer construction method for dynamic token selection according to claim 3, characterized in that: Taking the embedding representation of the token as input, optimizing the parameters of the auxiliary predictor includes: Inputting the current embedding representation of the token into the auxiliary predictor and outputting the probability value of the token being selected; Based on the output result of the auxiliary predictor, the parameters of the auxiliary predictor are optimized by a cross entropy loss function.

5. The decision mixer construction method for dynamic token selection according to claim 2, characterized in that: The decision mixer contains 2-4 layer groups, each of which contains at least one mixing layer and one standard Transformer layer; The standard Transformer layer is used to perform global attention calculation on the input sequence, stabilize the training process of the decision mixer and preserve global context information.

6. The decision mixer construction method for dynamic token selection according to claim 2, characterized in that: Optimizing the computational efficiency of the dynamic token selection mechanism includes: The length of the input sequence input to the attention calculation module is reduced to 30%-70% of the original length through a screening mechanism; The original embedding representations of the tokens that are not filtered are retained in the residual connection.

7. The decision mixer construction method for dynamic token selection according to claim 2, characterized in that: Dynamically optimizing and adjusting the dynamic token selection mechanism of the hybrid layer based on the output result of the auxiliary predictor, including: Predicting a probability value of the token being selected based on the embedding representation of the token at the current moment; The probability value of the token is compared with a preset threshold. If the probability value exceeds the preset threshold, the current mixed layer is activated to perform attention calculation, otherwise the token is directly passed.

8. The decision mixer construction method for dynamic token selection according to claim 1, characterized in that: The decision mixer is trained end-to-end, comprising: Obtaining a policy network and a history track of the decision mixer; the history track includes a state track, an action track, and a reward track; A loss function of the decision mixer is determined based on the policy network and the historical trajectory, and the decision mixer is trained with the goal of minimizing a function value of the loss function.

9. A decision mixer construction device for dynamic token selection, characterized in that: include: A building block for constructing a model framework for a decision mixer, wherein the decision mixer comprises alternating stacked mixing layers and standard Transformer layers, wherein the mixing layers include a dynamic token selection mechanism; A training module, configured to perform end-to-end training on the decision mixer and synchronously learn a binary probability distribution of token selection results based on the dynamic token selection mechanism through an auxiliary predictor; The inference module is used to dynamically optimize and adjust the dynamic token selection mechanism of the hybrid layer based on the output result of the auxiliary predictor to determine the final model framework of the decision mixer.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the decision mixer construction method for dynamic token selection according to any one of claims 1 to 8 is implemented.

Images