Transaction-generated pre-training transformer
By encoding and decoding lexical units in the interactive dataset, non-repeatable and repeatable lexical embeddings are generated. The input of the generative pre-trained Transformer is processed using vertical and horizontal Transformer modules, which solves the problem of excessive processing time caused by repeatable input and improves processing efficiency and performance.
CN122162127APending Publication Date: 2026-06-05VISA INTERNATIONAL SERVICE ASSOCIATION
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- VISA INTERNATIONAL SERVICE ASSOCIATION
- Filing Date
- 2024-08-14
- Publication Date
- 2026-06-05
Smart Images
Figure CN122162127A_ABST
Abstract
A method includes a computer obtaining interaction data comprising a plurality of interaction data sets. Each interaction data set comprises a plurality of tokens. The plurality of tokens includes non-repeating tokens and repeating tokens. For each interaction data set, the computer: encodes the non-repeating tokens of the plurality of tokens to form non-repeating token embeddings; retrieves repeating token embeddings from a memory using the repeating tokens, wherein the repeating token embeddings and the non-repeating token embeddings are in a plurality of first embeddings; concatenates the plurality of first embeddings with positional encodings to form second embeddings; and generates an interaction data set embedding based on the second embeddings. The computer determines one or more predicted interaction data set embeddings using the interaction data set embeddings, the interaction data set embeddings comprising the interaction data set embedding for each interaction data set. The computer decodes the one or more predicted interaction data set embeddings into predicted tokens.
Need to check novelty before this filing date? Find Prior Art