Transaction data vectorization and prediction method, model training method and equipment

Abstract

The embodiment of the invention provides a transaction data vectorization and prediction method, a model training method and equipment, and relates to the field of artificial intelligence. The method comprises the following steps: acquiring a transaction sequence of a target user, and performing dictionary coding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; performing high-dimensional mapping processing on each dictionary code to obtain a vector set; processing the vector set based on a self-attention model to obtain an output matrix corresponding to the vector set; and performing coding reduction processing on the output matrix to obtain a transaction data prediction result of the target user. According to the method, universal and unified vectorization expression of the transaction data can be realized, so that the effect of improving the transaction data prediction accuracy is achieved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a transaction data vectorization and prediction method, a model training method and a device. Background Art

[0002] In the process of financial institutions providing financial services to users, they need to predict users' transaction status in the future, which can effectively refine users' potential needs and generate high-quality "prompt words" for users.

[0003] In the prior art, based on a supervised learning model, manually annotated label data is used to learn and train a prediction model to predict the user's transaction status.

[0004] However, in the above method, manual labeling and screening of samples will affect the accuracy of labeling, resulting in insufficient model learning, and further resulting in inaccurate prediction results of the obtained prediction model. Summary of the invention

[0005] The embodiments of the present application provide a transaction data vectorization and prediction method, a model training method and a device, which are used to solve the technical problem that manual labeling and screening of samples will affect the accuracy of labeling, resulting in inaccurate prediction results of the obtained prediction model.

[0006] In a first aspect, an embodiment of the present application provides a transaction data vectorization and prediction method, including:

[0007] Acquire a transaction sequence of a target user, and perform dictionary coding on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the target user at a historical moment;

[0008] Performing high-dimensional mapping processing on each of the dictionary codes to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each of the dictionary codes;

[0009] Based on the self-attention model, the vector set is processed to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation between each transaction vector;

[0010] The output matrix is ​​coded and restored to obtain a transaction data prediction result of the target user; wherein the transaction data prediction result represents the transaction situation of the target user in a predicted future time period.

[0011] In a possible implementation, the vector set is processed based on a self-attention model to obtain an output matrix corresponding to the vector set, including:

[0012] Based on at least one self-attention mechanism of the self-attention model, determine a query matrix, a key matrix and a value matrix corresponding to the self-attention mechanism; wherein the query matrix includes each transaction vector currently being processed in the vector set; the key matrix includes all transaction vectors in the vector set; and the value matrix includes a value vector representing each transaction vector itself;

[0013] Based on each of the self-attention mechanisms, self-attention processing is performed on the query matrix, key matrix and value matrix corresponding to each of the self-attention mechanisms to obtain an output matrix corresponding to the vector set.

[0014] In a possible implementation, based on each of the self-attention mechanisms, self-attention processing is performed on the query matrix, the key matrix, and the value matrix corresponding to each of the self-attention mechanisms to obtain an output matrix corresponding to the vector set, including:

[0015] Determine a weight matrix corresponding to the self-attention mechanism according to the query matrix and the key matrix corresponding to the self-attention mechanism; wherein the weight matrix includes a weight coefficient corresponding to each of the transaction vectors;

[0016] According to the weight matrix, performing weighted summation processing on each value vector in the value matrix to obtain a self-attention matrix corresponding to the self-attention mechanism;

[0017] Each of the self-attention matrices is concatenated to obtain an output matrix corresponding to the vector set.

[0018] In a possible implementation, determining a weight matrix corresponding to the self-attention mechanism according to a query matrix and a key matrix corresponding to the self-attention mechanism includes:

[0019] Determine a similarity matrix between the query matrix and the key matrix; wherein the similarity matrix represents the correlation between each transaction vector and other transaction vectors except the transaction vector;

[0020] Scaling the similarity matrix to obtain a scaled similarity matrix;

[0021] The scaled similarity matrix is ​​normalized to obtain a weight matrix corresponding to the self-attention mechanism.

[0022] In a possible implementation, performing encoding restoration processing on the output matrix to obtain a transaction data prediction result of the target user includes:

[0023] Performing dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix; wherein the predicted transaction sequence includes predicted dictionary codes at N future moments within a preset time period; the predicted dictionary codes represent transaction data generated at the predicted future moments;

[0024] The prediction dictionary codes are subjected to field restoration and splicing processing to obtain the transaction data prediction result; wherein the transaction data prediction result includes predicted transaction messages at N future moments; the predicted transaction messages represent the transaction status of the target user at the predicted future moment.

[0025] In a possible implementation, performing dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix includes:

[0026] Performing dictionary coding restoration processing on the i-th output matrix to obtain the i-th prediction dictionary code in the predicted transaction sequence;

[0027] Generate a new transaction vector corresponding to the i-th prediction dictionary code; and add the new transaction vector corresponding to the i-th prediction dictionary code to the vector set;

[0028] Based on the self-attention model, the vector set is processed to obtain an i+1th output matrix;

[0029] The i+1th output matrix is ​​subjected to dictionary coding restoration processing to obtain the i+1th predicted dictionary code in the predicted transaction sequence.

[0030] In a possible implementation, performing dictionary coding restoration processing on the i-th output matrix to obtain the i-th prediction dictionary code in the predicted transaction sequence includes:

[0031] Performing dictionary coding restoration processing on the i-th output matrix to obtain at least one restored dictionary code corresponding to the i-th output matrix; wherein the restored dictionary code has a prediction probability, and the prediction probability is a generation probability of the predicted transaction data;

[0032] The restored dictionary code with the highest prediction probability is determined from at least one restored dictionary code corresponding to the i-th output matrix as the i-th predicted dictionary code in the predicted transaction sequence.

[0033] In a possible implementation, performing field restoration and splicing processing on each of the prediction dictionary codes to obtain the transaction data prediction result includes:

[0034] Performing field restoration processing on the prediction dictionary code to obtain at least one prediction message field corresponding to the prediction dictionary code; wherein the prediction message field represents a transaction field in the transaction data generated at a predicted future moment;

[0035] Performing mapping processing on at least one predicted message field corresponding to the predicted dictionary code to obtain at least one predicted message value corresponding to the predicted dictionary code;

[0036] At least one predicted message value corresponding to the predicted dictionary code is concatenated to obtain a predicted transaction message corresponding to the predicted dictionary code.

[0037] In a possible implementation, performing dictionary coding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message includes:

[0038] Discretization processing is performed on each message field in the transaction message to obtain a discretized value corresponding to each message field; wherein the message field represents a transaction field in the transaction data;

[0039] Perform segmentation and mapping processing on the discretized value to obtain a field value corresponding to the message field;

[0040] The values ​​of each field are concatenated to obtain a dictionary code corresponding to the transaction message.

[0041] In a possible implementation, each of the dictionary codes is subjected to high-dimensional mapping processing to obtain a vector set, including:

[0042] Performing one-hot numbering processing on the dictionary code to obtain a one-hot number corresponding to the transaction message;

[0043] A high-dimensional mapping process is performed on the unique hot number to obtain a transaction vector corresponding to the transaction message.

[0044] In a second aspect, an embodiment of the present application provides a model training method for transaction data vectorization and prediction, including:

[0045] Acquire a transaction sequence of the user to be trained, and perform dictionary coding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the user to be trained at a historical moment;

[0046] Performing high-dimensional mapping processing on each of the dictionary codes to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each of the dictionary codes;

[0047] Based on the initial model, the vector set is processed to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation relationship between each transaction vector;

[0048] The initial model is trained according to the output matrix to obtain a self-attention model; wherein the self-attention model is used to process the transaction sequence of the target user as described in the first aspect to obtain a transaction data prediction result of the target user; the transaction data prediction result represents the transaction status of the target user in a predicted future time period.

[0049] In a possible implementation, the vector set is processed based on the initial model to obtain an output matrix corresponding to the vector set, including:

[0050] Based on at least one self-attention mechanism of the self-attention model, determine a query matrix, a key matrix and a value matrix corresponding to the self-attention mechanism; wherein the query matrix includes each transaction vector currently being processed in the vector set; the key matrix includes all transaction vectors in the vector set; and the value matrix includes a value vector representing each transaction vector itself;

[0051] Based on each of the self-attention mechanisms, self-attention processing is performed on the query matrix, key matrix and value matrix corresponding to each of the self-attention mechanisms to obtain an output matrix corresponding to the vector set.

[0052] In a possible implementation, based on each of the self-attention mechanisms, self-attention processing is performed on the query matrix, the key matrix, and the value matrix corresponding to each of the self-attention mechanisms to obtain an output matrix corresponding to the vector set, including:

[0053] Determine a weight matrix corresponding to the self-attention mechanism according to the query matrix and the key matrix corresponding to the self-attention mechanism; wherein the weight matrix includes a weight coefficient corresponding to each of the transaction vectors;

[0054] According to the weight matrix, performing weighted summation processing on each value vector in the value matrix to obtain a self-attention matrix corresponding to the self-attention mechanism;

[0055] Each of the self-attention matrices is concatenated to obtain an output matrix corresponding to the vector set.

[0056] In a possible implementation, determining a weight matrix corresponding to the self-attention mechanism according to a query matrix and a key matrix corresponding to the self-attention mechanism includes:

[0057] Determine a similarity matrix between the query matrix and the key matrix; wherein the similarity matrix represents the correlation between each transaction vector and other transaction vectors except the transaction vector;

[0058] Scaling the similarity matrix to obtain a scaled similarity matrix;

[0059] The scaled similarity matrix is ​​normalized to obtain a weight matrix corresponding to the self-attention mechanism.

[0060] In a possible implementation, the initial model is trained according to the output matrix to obtain a self-attention model, including:

[0061] The output matrix is ​​encoded and restored to obtain a transaction prediction result of the user to be trained; wherein the transaction prediction result represents the transaction situation of the user to be trained within a preset time period of the prediction;

[0062] The initial model is trained according to the transaction prediction results and actual transaction results of the user to be trained to obtain the self-attention model; wherein the actual transaction results represent the transaction status of the user to be trained within the actual preset time period.

[0063] In a possible implementation, performing encoding restoration processing on the output matrix to obtain the transaction prediction result of the user to be trained includes:

[0064] Performing dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix; wherein the predicted transaction sequence includes predicted dictionary codes at M preset moments within a preset time period; the predicted dictionary codes represent transaction data generated at the predicted preset moments;

[0065] Field restoration and splicing processing is performed on each of the prediction dictionary codes to obtain the transaction data prediction result; wherein the transaction data prediction result includes predicted transaction messages at M preset moments; the predicted transaction message represents the transaction status of the user to be trained at the predicted preset moment.

[0066] In a possible implementation, performing dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix includes:

[0067] Performing dictionary coding restoration processing on the j-th output matrix to obtain the j-th prediction dictionary code in the predicted transaction sequence;

[0068] Generate a new transaction vector corresponding to the j-th prediction dictionary code; and add the new transaction vector corresponding to the j-th prediction dictionary code to the vector set;

[0069] Based on the self-attention model, the vector set is processed to obtain the j+1th output matrix;

[0070] The j+1th output matrix is ​​subjected to dictionary coding restoration processing to obtain the j+1th predicted dictionary code in the predicted transaction sequence.

[0071] In a possible implementation, performing dictionary coding restoration processing on the j-th output matrix to obtain the j-th prediction dictionary code in the predicted transaction sequence includes:

[0072] Performing dictionary coding restoration processing on the j-th output matrix to obtain at least one restored dictionary code corresponding to the j-th output matrix; wherein the restored dictionary code has a prediction probability, and the prediction probability is a generation probability of the predicted transaction data;

[0073] The restored dictionary code with the highest prediction probability is determined from at least one restored dictionary code corresponding to the j-th output matrix as the j-th predicted dictionary code in the predicted transaction sequence.

[0074] In a possible implementation, performing field restoration and splicing processing on each of the prediction dictionary codes to obtain the transaction data prediction result includes:

[0075] Performing field restoration processing on the prediction dictionary code to obtain at least one prediction message field corresponding to the prediction dictionary code; wherein the prediction message field represents a transaction field in the transaction data generated at a preset time of prediction;

[0076] Performing mapping processing on at least one predicted message field corresponding to the predicted dictionary code to obtain at least one predicted message value corresponding to the predicted dictionary code;

[0077] At least one predicted message value corresponding to the predicted dictionary code is concatenated to obtain a predicted transaction message corresponding to the predicted dictionary code.

[0078] In a possible implementation, performing dictionary coding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message includes:

[0079] Discretization processing is performed on each message field in the transaction message to obtain a discretized value corresponding to each message field; wherein the message field represents a transaction field in the transaction data;

[0080] Perform segmentation and mapping processing on the discretized value to obtain a field value corresponding to the message field;

[0081] The values ​​of each field are concatenated to obtain a dictionary code corresponding to the transaction message.

[0082] In a possible implementation, each of the dictionary codes is subjected to high-dimensional mapping processing to obtain a vector set, including:

[0083] Performing one-hot numbering processing on the dictionary code to obtain a one-hot number corresponding to the transaction message;

[0084] A high-dimensional mapping process is performed on the unique hot number to obtain a transaction vector corresponding to the transaction message.

[0085] In a third aspect, an embodiment of the present application provides a transaction data vectorization and prediction device, including:

[0086] An encoding module, used to obtain a transaction sequence of a target user, and perform dictionary encoding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the target user at a historical moment;

[0087] A mapping module, used for performing high-dimensional mapping processing on each of the dictionary codes to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each of the dictionary codes;

[0088] A processing module, configured to process the vector set based on a self-attention model to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation between each transaction vector;

[0089] The restoration module is used to perform encoding restoration processing on the output matrix to obtain the transaction data prediction result of the target user; wherein the transaction data prediction result represents the transaction situation of the target user in the predicted future time period.

[0090] In a fourth aspect, an embodiment of the present application provides a model training device for transaction data vectorization and prediction, including:

[0091] An encoding module, used to obtain a transaction sequence of a user to be trained, and to perform dictionary encoding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the user to be trained at a historical moment;

[0092] A mapping module, used for performing high-dimensional mapping processing on each of the dictionary codes to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each of the dictionary codes;

[0093] A processing module, used to process the vector set based on the initial model to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation relationship between each transaction vector;

[0094] A training module is used to train the initial model according to the output matrix to obtain a self-attention model; wherein the self-attention model is used to process the transaction sequence of the target user as described in the third aspect to obtain the transaction data prediction result of the target user; the transaction data prediction result represents the transaction status of the target user in the predicted future time period.

[0095] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0096] The memory stores computer-executable instructions;

[0097] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method of the first aspect and / or the second aspect as described above.

[0098] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-readable storage medium is stored computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method of the first aspect and / or the second aspect above.

[0099] In a seventh aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements the method of the first aspect and / or the second aspect above when executed by a processor.

[0100] The transaction data vectorization and prediction method, model training method and device provided in the embodiments of the present application perform dictionary encoding on each transaction data of the user, and then project it into a transaction vector in a high-dimensional space, and input it into a self-attention model to process all transaction vectors to obtain an output matrix. The correlation between each transaction vector can be found, and the transaction status of the target user in the future time period can be predicted through the encoding and restoration processing of the output matrix; furthermore, a universal and unified vectorized expression of transaction data can be achieved, and through the processing of the self-attention model, the correlation between each transaction vector can be found to perform transaction data prediction, which can improve the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0102] Figure 1 A schematic diagram of an application scenario provided for this application;

[0103] Figure 2 A flowchart of a transaction data quantization and prediction method provided in an embodiment of the present application;

[0104] Figure 3 A framework diagram of a transaction prediction model based on generative artificial intelligence provided in an embodiment of the present application;

[0105] Figure 4 A schematic diagram of another transaction data vectorization and prediction method provided in an embodiment of the present application;

[0106] Figure 5 A transaction prediction timing diagram provided in an embodiment of the present application;

[0107] Figure 6 A flowchart of a model training method for transaction data vectorization and prediction provided in an embodiment of the present application;

[0108] Figure 7 A flowchart of another model training method for transaction data vectorization and prediction provided in an embodiment of the present application;

[0109] Figure 8 A schematic diagram of the structure of a transaction data vectorization and prediction device provided in an embodiment of the present application;

[0110] Fig. 9 A schematic diagram of the structure of a model training device for transaction data vectorization and prediction provided in an embodiment of the present application;

[0111] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0112] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0113] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0114] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0115] In addition, this application involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and the use of artificial intelligence technology for automated decision-making, and a technical solution for making decisions that have a significant impact on personal rights and interests based on the results of automated decision-making. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decision-making; if the user chooses to reject, the expert decision-making process will be entered.

[0116] It should be noted that the transaction data vectorization and prediction method, model training method and equipment of the present application can be used in the field of artificial intelligence, and can also be used in any field other than artificial intelligence. The application field of the transaction data vectorization and prediction method, model training method and equipment of the present application is not limited.

[0117] Figure 1 A schematic diagram of an application scenario provided by this application. Figure 1 As shown, the specific application scenario of the present application includes an electronic device 101 and a server 102. The electronic device 101 processes from the server 102, obtains the transaction sequence of each target user, and performs prediction processing on multiple transaction data in the transaction sequence to obtain a transaction data prediction result.

[0118] In the process of financial institutions providing financial services to users, they need to predict users' transaction status in the future, which can effectively refine users' potential needs and generate high-quality "prompt words" for users.

[0119] In one example, based on supervised learning models, such as logistic regression models, tree models (random forests), ensemble models, etc., manually annotated labeled data is used to learn the function mapping from input variable X to output variable Y. Through associated samples and labels, the machine can learn the mapping relationship between input and output, obtain a prediction model, and use it for transaction data prediction processing.

[0120] For example, for transaction anti-fraud, it is usually based on the accumulated historical sample data, and statistical features and labels are extracted by classification, including transaction media, transaction channels, transaction amounts, transaction locations, transaction times, etc. Then, supervised training is carried out using machine learning models to obtain a prediction model, which is used for predicting and processing transaction data of target users.

[0121] However, in the above methods, on the one hand, feature engineering based on manual experience is limited by professional knowledge and may miss important features, which in turn affects the prediction accuracy and generalization ability of the model; on the other hand, samples need to be manually labeled, and the process of manually labeling samples is itself a subjective and error-prone task. The professional level of the labeler, understanding bias and other factors may affect the accuracy of the labeling, and the lack of labeled data may lead to insufficient model learning. In addition, models need to be designed for specific tasks or specific scenarios. The adaptation scenarios of a single model are relatively limited, and the model management cost is high.

[0122] In response to the above technical problems, this application proposes the following technical concept: by dictionary encoding each transaction data of the user, and then projecting it into a transaction vector in a high-dimensional space, and inputting it into the self-attention model to process all transaction vectors and obtain an output matrix, the correlation between each transaction vector can be found, and through the encoding and restoration process of the output matrix, the transaction status of the target user in the future time period can be predicted. In this way, the technical problem existing in the prior art that manual labeling and screening of samples will affect the accuracy of labeling, resulting in inaccurate prediction results of the obtained prediction model, can be solved.

[0123] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0124] Figure 2 A flowchart of a transaction data quantization and prediction method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method includes:

[0125] 201. Obtain a transaction sequence of a target user, and perform dictionary coding on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the target user at a historical moment.

[0126] Exemplarily, the execution subject of this embodiment may be an electronic device, hereinafter referred to as a device. The device may directly obtain the transaction sequence of the target user to be predicted from a local database or from an external server, wherein the transaction sequence includes multiple transaction messages, i.e., transaction data representing the target user at a historical moment, such as transaction records and transaction logs at multiple consecutive historical moments. Figure 3 A framework diagram of a transaction prediction model based on generative artificial intelligence provided in an embodiment of the present application, such as Figure 3 As shown, the device calls a preset transaction vector encoder, which includes a transaction dictionary mapping sublayer and a transaction vector embedding layer. Based on the transaction dictionary mapping sublayer, dictionary encoding is performed on each transaction message to obtain a dictionary code corresponding to each transaction message to correspond to each original transaction data.

[0127] For example, a transaction can be represented by a string of numbers, that is, a corresponding transaction dictionary code is generated. Taking a transaction of user A as an example, the corresponding transaction dictionary code is '21100040'.

[0128] 202. Perform high-dimensional mapping processing on each dictionary code to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each dictionary code.

[0129] For example, in combination Figure 3 ,Based on the transaction vector embedding sublayer, each dictionary code obtained, such as each string of numbers, is projected into a high-dimensional space to generate a vector, and the transaction vector corresponding to each dictionary code is obtained to achieve a universal representation of transactions, and then a vector set is obtained to be processed.

[0130] 203. Based on the self-attention model, the vector set is processed to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation between each transaction vector.

[0131] For example, in combination Figure 3 , the device calls a preset autoregressive decoder, in which a trained self-attention model is deployed, which may include four sublayers, namely, a self-attention sublayer, a residual connection and normalization sublayer, a forward connection layer, and a residual connection and normalization sublayer. Based on the self-attention sublayer, self-attention processing is performed on each transaction vector in the obtained vector set, and through the processing of the residual connection and normalization sublayer, the forward connection layer, and the residual connection and normalization sublayer, the correlation relationship between each transaction vector in the vector set is identified. The output result is represented in matrix form, and then the output matrix corresponding to the vector set is obtained.

[0132] 204. Perform encoding and restoration processing on the output matrix to obtain a transaction data prediction result of the target user; wherein the transaction data prediction result represents the transaction situation of the target user in the predicted future time period.

[0133] For example, in combination Figure 3 The device calls a preset transaction generator, and based on the transaction vector restoration sublayer of the transaction generator, performs encoding restoration processing on the obtained output matrix, that is, restores the output matrix into corresponding multiple vectors, and based on the transaction dictionary mapping sublayer of the transaction generator, performs mapping processing on the restored multiple vectors to obtain prediction data, that is, transaction data prediction results, and then predicts the transaction situation of the target user in the future time period, such as whether the target user has transactions at multiple future moments in the future time period, or whether the target user's transaction items increase in the future time period, or the transaction data and transaction records generated when the target user trades in the future time period.

[0134] In this embodiment, a transaction data vectorization and prediction method is provided, in which each transaction data of the user is encoded into a transaction vector through a dictionary, and is input into a self-attention model to process all transaction vectors to obtain an output matrix to discover the correlation between each transaction vector, and through the encoding and restoration process of the output matrix, the transaction status of the target user in the future time period is predicted; furthermore, a universal and unified vectorized expression of transaction data can be achieved, and through the processing of the self-attention model, the correlation between each transaction vector is discovered to perform transaction data prediction, which can improve the prediction accuracy.

[0135] Figure 4 A flow chart of another transaction data vectorization and prediction method provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the method includes:

[0136] 301. Obtain a transaction sequence of a target user, and perform dictionary coding on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the target user at a historical moment.

[0137] Exemplarily, this step may refer to step 201 and will not be described in detail here.

[0138] In one example, step 301 includes the following steps:

[0139] The first step of step 301 is to discretize each message field in the transaction message to obtain a discretized value corresponding to each message field; wherein the message field represents a transaction field in the transaction data.

[0140] The second step of step 301 is to perform segmentation mapping processing on the discretized values ​​to obtain field values ​​corresponding to the message fields.

[0141] The third step of step 301 is to concatenate the values ​​of each field to obtain the dictionary code corresponding to the transaction message.

[0142] For example, each transaction message includes multiple message fields, such as transaction amount, time, date, card attribute, response code, merchant type, etc. Figure 3 Based on the transaction dictionary mapping layer, the continuous space of each field in the transaction message, such as transaction amount, time, date, card attribute, response code, merchant type, transaction area, transaction type, etc., is discretized and mapped into a discrete value through interval mapping. For the i-th field Domaini in the transaction message, its corresponding value space is [buck i_0 ,…,buck i_n ,…,bucki_N i+1 ]Total N i Perform segmentation mapping to obtain the corresponding field value Domain_vec i =f(Domain)=0when buck i_0 ≤Domain i ≤buck i_1 ,…,n when buck i_n ≤Domain i ≤buck i_n+1 ,…,N i when buck i_Ni ≤Domain i ≤buck i_Ni+1 , concatenate the field values ​​in the transaction message to obtain the dictionary code Trans_vec of each transaction message i =concat(Domain_vec0, ..., Domain_veci, ..., Domain_vecM), where M is the number of fields in the transaction message that are mapped to dictionary encoding. The mapping logic of different transaction message fields is determined based on business understanding.

[0143] For example, the transaction amount between 0 and 1 yuan is mapped to 0, 1 to 10 yuan is mapped to 1, and 10 to 100 yuan is mapped to 2; the transaction time between 7:00 and 11:00 is mapped to 0, the transaction time between 11:00 and 13:00 is mapped to 1, and so on, the debit card is mapped to 0, and the credit card is mapped to 1; the success response code is mapped to 0, and the failure response code is mapped to 1; the merchant type is mapped to 0 if it is an offline retail store, and mapped to 1 if it is a hospital; and so on. After all fields are mapped, a transaction can be represented by a string of numbers, that is, the corresponding transaction dictionary code is generated. Taking a transaction of user A as an example, its corresponding transaction dictionary code is '21100040', as shown in the following table.

[0144]

[0145] 302. Perform high-dimensional mapping processing on each dictionary code to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each dictionary code.

[0146] Exemplarily, this step may refer to step 202 and will not be described in detail here.

[0147] In one example, step 302 includes the following steps:

[0148] The first step of step 302 is to perform one-hot numbering processing on the dictionary code to obtain a one-hot number corresponding to the transaction message.

[0149] The second step of step 302 is to perform high-dimensional mapping processing on the unique hot number to obtain a transaction vector corresponding to the transaction message.

[0150] For example, in combination Figure 3 , based on the transaction vector embedding layer, each dictionary code obtained by the transaction dictionary mapping sublayer is numbered as "one-hot" and converted into a transaction vector. First, the space size of each dictionary code Trans_vec is Assign each dictionary code of Trans_vec to 0 to The unique serial number of is the "one hot" number. Considering that some values ​​in the entire dictionary space do not match the real transaction data, and if the transaction dictionary space is too large, it may cause the model to converge slowly. Therefore, the S serial numbers in the transaction dictionary space that do not match the real transaction data are removed to form a size of Dictionary space, in other words, all dictionary codes are divided into class; secondly, map each dictionary-encoded "unique hot" number to a vector space with dimension dmodel to obtain the transaction vector corresponding to the transaction message, which is convenient for subsequent deep self-attention decoder to process. This process is called "vector embedding", and the specific value of the transaction vector can be obtained through model learning.

[0151] For example, taking the transaction sequence of user A as an example, the transaction dictionary after the transaction dictionary mapping sublayer is encoded as ['21100040', '11101030', '01100140', '02100030'], and the corresponding "one hot" number sequence is [13, 43, 4567, 3443]. After the transaction vector embedding sublayer, each "one hot" number is mapped to a vector of the form [0.12, 0.34, ..., 0.11, ...] and input into the deep self-attention neural network decoder.

[0152] 303. Based on at least one self-attention mechanism of the self-attention model, determine a query matrix, a key matrix and a value matrix corresponding to the self-attention mechanism; wherein the query matrix includes each transaction vector currently being processed in the vector set; the key matrix includes all transaction vectors in the vector set; and the value matrix includes a value vector representing each transaction vector itself.

[0153] Exemplarily, the self-attention model deployed by the device is an autoregressive decoder based on a deep self-attention neural network, combined with Figure 3 Based on the self-attention sub-layer, in order to more effectively expand the model's ability to pay attention to different attributes of transaction vectors, a multi-head self-attention mechanism is deployed. For each self-attention mechanism, according to each transaction vector in the vector set, the query matrix corresponding to the transaction vector currently being processed under the self-attention mechanism is determined, as well as the key matrix including all transaction vectors in the vector set, and the value matrix including the value vector representing each transaction vector itself.

[0154] For example, in the attention mechanism, query, key and value are important components for calculating the correlation between different vectors. For a vector set, including three transaction vectors: A, B and C, for each self-attention mechanism, including each currently processed transaction vector, it is represented as the initial query matrix Q, all transaction vectors (A, B, C) in the vector set will be represented as initial key vectors K1, K2, K3, and the initial key matrix K will be obtained. All transaction vectors (A, B, C) in the vector set will also be represented as initial value vectors V1, V2, V3, and the initial value matrix V will be obtained. Based on the multi-head self-attention mechanism, it needs to be processed in each self-attention mechanism. Therefore, the initial query matrix Q, the initial key matrix K, and the initial value matrix V need to be dimensionally scaled to obtain the query matrix QW under the i-th self-attention mechanism. i Q , key matrix KW i K , value matrix VW i V , where W i Q is the preset matrix used to scale the dimension of Q, W i K is the preset matrix used to scale the dimension of K, W i V is a preset matrix used to scale the dimension of V; for example, the dimension of Q is 200, the number of attention heads is 4, and W i Q The dimension dmodel*dq is 200*50, and QW is obtained i Q , the QW i Q The dimension of is 50, the dimension of K is 200, and the dimension of W is i K The dimension dmodel*d K 200*50, we get KW i K , the KW i K The dimension of is 50, the dimension of V is 200, and the dimension of W is i V The dimension dmodel*d V is 200*50, and we get VW i V , the VW i V The dimension of dmodel is 50; dmodel is the dimension of the transaction vector, dq, d K ,d V It is determined by the dimension of the transaction vector and the number of attention heads.

[0155] 304. Based on the respective attention mechanisms, self-attention processing is performed on the query matrix, key matrix, and value matrix corresponding to the respective attention mechanisms to obtain an output matrix corresponding to the vector set.

[0156] Exemplarily, through each self-attention mechanism, self-attention processing is performed on the query matrix, key matrix and value matrix corresponding to the self-attention mechanism. For example, the query matrix and key matrix can be processed by the self-attention mechanism to obtain the correlation between different transaction vectors, and then the respective value vectors in the value matrix are used to determine the final representation of the current transaction vector, thereby obtaining the output matrix corresponding to all transaction vectors corresponding to the self-attention mechanism.

[0157] In one example, step 304 includes the following steps:

[0158] The first step of step 304 is to determine the weight matrix corresponding to the self-attention mechanism according to the query matrix and the key matrix corresponding to the self-attention mechanism; wherein the weight matrix includes the weight coefficient corresponding to each transaction vector.

[0159] The second step of step 304 is to perform weighted summation processing on each value vector in the value matrix according to the weight matrix to obtain a self-attention matrix corresponding to the self-attention mechanism.

[0160] The third step of step 304 is to concatenate the respective attention matrices to obtain an output matrix corresponding to the vector set.

[0161] Exemplarily, through each self-attention mechanism, the query matrix and key matrix corresponding to the self-attention mechanism are calculated for self-attention weights to obtain each transaction vector, that is, the weight coefficient corresponding to each currently processed transaction vector, and then the weight coefficients corresponding to all transaction vectors are combined to obtain the weight matrix corresponding to the self-attention mechanism. Based on each self-attention mechanism, the weight coefficients of the weight matrix and the value vectors in the value matrix are weighted and summed to obtain the self-attention matrix corresponding to all transaction vectors under the self-attention mechanism. Through the splicing operation, the self-attention matrices corresponding to each self-attention mechanism are spliced ​​tightly to obtain the output matrix; for example, MutiHead(Q,K,V)=Concat(head1,…,headi,…,headh)W 0 , where headi is the self-attention matrix Attention(QW corresponding to the i-th self-attention mechanism i Q ,KW i K ,VW i V ), W 0is a preset matrix of dimension hdv*dmodelde. Furthermore, in this way, the attention mechanism can dynamically adjust the contribution of each transaction vector in the final representation according to the correlation between transaction vectors, thereby capturing richer contextual information.

[0162] For example, for a vector set, including three transaction vectors, for the current transaction vector, under the i-th self-attention mechanism, for the query matrix QW containing each current transaction vector i Q , the key matrix KW of the vector set i K , the value matrix VW of the vector set i V , calculate the query matrix QW i Q With the key matrix KW i K The correlation QW of each key vector in i Q KW i KT , and normalize it to get the attention weight matrix corresponding to the vector set, which includes the weight coefficient corresponding to each transaction vector. For example, the weight coefficient of transaction vector A includes weight 1 as 0.7, corresponding weight 2 as 0.15, and corresponding weight 3 as 0.15, and then get; Use the attention weight coefficients in the attention weight matrix to get the value matrix VW i V The weighted sum of each value vector in is performed to obtain the final representation of each transaction vector, such as the final representation of transaction vector A = 0.7*V1+0.15*V2+0.15*V3, and then the output matrix corresponding to all transaction vectors under each self-attention mechanism is obtained; further, in this way, the self-attention mechanism can dynamically adjust the contribution of each transaction vector in the final representation according to the correlation between the transaction vectors, thereby capturing richer contextual information.

[0163] In one example, the first step of step 304 includes:

[0164] Step 1: Determine a similarity matrix between the query matrix and the key matrix; wherein the similarity matrix represents the correlation between each transaction vector and other transaction vectors except the transaction vector.

[0165] Step 2: Scaling the similarity matrix to obtain a scaled similarity matrix.

[0166] Step 3: Normalize the scaled similarity matrix to obtain the weight matrix corresponding to the self-attention mechanism.

[0167] Specifically, through each self-attention mechanism, the query matrix QW corresponding to the self-attention mechanism is i Q , key matrix KW i K Perform similarity calculation to obtain the similarity, i.e., correlation, between each transaction vector being processed and the remaining transaction vectors except the transaction vector, and then combine to obtain the similarity matrix QW between the query matrix and the key matrix i Q (KW i K ) T ; Since the dimensions of the query matrix and the key matrix may be large, they need to be scaled, which helps to avoid the problem of gradient vanishing or gradient exploding when calculating the Softmax function, and then obtain the scaled similarity matrix The scaled similarity matrix is ​​normalized by the normalization algorithm of the self-attention mechanism, such as the Softmax function, to obtain the weight matrix corresponding to the self-attention mechanism. It is worth adding that based on the above process, the weight matrix Afterwards, through each self-attention mechanism, the self-attention matrix corresponding to the i-th self-attention mechanism is obtained

[0168]

[0169] 305. Perform dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix; wherein the predicted transaction sequence includes predicted dictionary codes at N future moments within a preset time period; and the predicted dictionary codes represent transaction data generated at the predicted future moments.

[0170] For example, in combination Figure 3 Based on the transaction vector restoration sublayer, the output matrix of the autoregressive decoder is restored through a linear layer to the predicted dictionary code at N future moments within a preset time period, that is, the transaction data generated at each future moment in the prediction, and then the predicted transaction sequence is obtained.

[0171] In one example, step 305 includes the following steps:

[0172] The first step of step 305 is to perform dictionary coding restoration processing on the ith output matrix to obtain the ith prediction dictionary code in the predicted transaction sequence.

[0173] The second step of step 305 is to generate a new transaction vector corresponding to the ith prediction dictionary code; and add the new transaction vector corresponding to the ith prediction dictionary code to the vector set.

[0174] The third step of step 305 is to process the vector set based on the self-attention model to obtain the i+1th output matrix.

[0175] The fourth step of step 305 is to perform dictionary coding restoration processing on the i+1th output matrix to obtain the i+1th predicted dictionary code in the predicted transaction sequence.

[0176] For example, in combination Figure 3 , based on the transaction vector restoration sublayer, for the first output matrix, a linear layer is used to perform dictionary coding restoration processing, and the prediction dictionary coding at the first future moment in the future time period is restored, and the first prediction dictionary coding is input into the transaction vector encoder to perform vector mapping processing on the first prediction dictionary coding to obtain a new transaction vector of the first prediction dictionary coding, that is, the first new transaction vector, and the first new transaction vector is input into the vector set, that is, input into the initially obtained vector set to obtain a new vector set, and the new vector set is input into the self-attention model in the autoregressive decoder. In the model, processing is performed, and the processing process is consistent with the aforementioned processing process to obtain the second output matrix. Based on the transaction vector restoration sublayer, for the second output matrix, dictionary coding restoration processing is performed through a linear layer to restore it to the predicted dictionary coding at the second future moment in the preset time period. The above process is repeated to obtain the i-th output matrix. Through the dictionary coding restoration processing, vector mapping, adding vector sets, and self-attention model processing in the aforementioned process, the i+1-th output matrix is ​​obtained, and then the i+1-th predicted dictionary coding is obtained after dictionary coding restoration processing to obtain the final predicted transaction sequence.

[0177] For example, Figure 5 A transaction prediction timing diagram provided in an embodiment of the present application, such as Figure 5 As shown, given the transaction data of the first n transactions in a transaction sequence, the model completes the prediction of the transaction data of the n+1th transaction (at time t1), and inputs the transaction data of the n+1th transaction into the model again, and the prediction of the transaction data of the n+2th transaction (at time t2) can be achieved. By repeating this process for multiple rounds, the generation of a predicted transaction sequence of autoregressive transactions can be achieved.

[0178] For example, taking the transaction sequence of user A, the first five transactions of user A are [13, 43, 4567, 3443, 3456]. The model generates the sixth transaction '2387' based on these five transactions [13, 43, 4567, 3443, 3456]. Then [13, 43, 4567, 3443, 3456, 2387] is fed into the model to generate the seventh transaction '9875', and so on, eventually generating a complete transaction sequence [13, 43, 4567, 3443, 3456, 2387, 24566, …].

[0179] In one example, the first step of step 305 includes:

[0180] Step 1: Perform dictionary coding restoration processing on the ith output matrix to obtain at least one restored dictionary code corresponding to the ith output matrix; wherein the restored dictionary code has a prediction probability, and the prediction probability is the generation probability of the predicted transaction data.

[0181] Step 2: Determine the restoration dictionary code with the highest prediction probability from at least one restoration dictionary code corresponding to the ith output matrix as the ith prediction dictionary code in the predicted transaction sequence.

[0182] Specifically, combined Figure 3 , for the i-th output matrix, based on the transaction vector restoration sublayer, the i-th output matrix is ​​subjected to dictionary coding restoration processing to obtain multiple restored dictionary codes corresponding to the i-th output matrix, each restored dictionary code has a prediction probability obtained by the self-attention model, that is, the generation probability of the predicted transaction data corresponding to the restored dictionary code; from all the restored dictionary codes, the restored dictionary code with the largest prediction probability is selected, and the restored dictionary code is determined as the i-th predicted dictionary code in the predicted transaction sequence, that is, the prediction result of the next transaction that will occur in the future.

[0183] 306. Perform field restoration and splicing processing on each prediction dictionary code to obtain a transaction data prediction result; wherein the transaction data prediction result includes predicted transaction messages at N future moments; the predicted transaction message represents the transaction status of the target user at the predicted future moment.

[0184] For example, in combination Figure 3 Based on the transaction dictionary mapping sublayer, the fields of each prediction dictionary code are restored to obtain the corresponding predicted transaction message, that is, the transaction status of the target user at each future moment, and all predicted transaction messages are spliced ​​and processed to obtain the transaction data prediction result.

[0185] In one example, step 306 includes the following steps:

[0186] The first step of step 306 is to perform field restoration processing on the prediction dictionary code to obtain at least one prediction message field corresponding to the prediction dictionary code; wherein the prediction message field represents a transaction field in the transaction data generated at the predicted future moment.

[0187] The second step of step 306 is to perform mapping processing on at least one predicted message field corresponding to the predicted dictionary code to obtain at least one predicted message value corresponding to the predicted dictionary code.

[0188] The third step of step 306 is to perform concatenation processing on at least one predicted message value corresponding to the predicted dictionary code to obtain a predicted transaction message corresponding to the predicted dictionary code.

[0189] For example, in combination Figure 3 Based on the transaction dictionary mapping sublayer, each prediction dictionary code is restored to the real multiple prediction message fields corresponding to each prediction dictionary code, such as transaction amount, transaction type and other information, through Domain_veci1=f -1 (Domain_veci1), map each predicted message field to the actual transaction message value, that is, the predicted message value, concatenate all the predicted message values ​​corresponding to each predicted dictionary code, and obtain the predicted transaction message corresponding to each predicted dictionary code.

[0190] For example, taking the transaction sequence of user A as an example, [13, 43, 4567, 3443, 3456, 2387, 24566, ...] is mapped to ['21100040', '11101030', '01100140', '02100030', '24500040', '16701030', '01112140', ..., '02189530'], and then through Domain veci1 =f -1 (Domain veci1), mapped to detailed transaction information, that is, the predicted transaction data of the user in the future period is obtained. It is worth adding that after generating the predicted transaction of the user in the future period, the application can be carried out from the two scenarios of user operation and risk management. In the user operation scenario, the general large model is used to assist in writing financial service solutions. However, how to understand the real needs of each customer and propose accurate "prompt words" to the general large model is a difficulty in implementation. This embodiment can effectively refine the potential needs of users by generating a transaction sequence for a period of time in the future, and help banks generate high-quality "prompt words" for each of their customers as input for docking with the general large model, truly realizing "one customer, one solution"; in the risk management scenario, based on the real-time risk control system, the dynamic decision-making model or strategy in the process is initially realized through the machine learning model. This embodiment can greatly help banks improve risk management effectiveness and improve business processing efficiency by analyzing the deviation between actual transactions and predicted transactions.

[0191] In this embodiment, based on the above embodiments, on the one hand, the original transaction is dictionary encoded and then projected into a vector in a high-dimensional space to achieve a universal representation of the transaction, and there is no need to design feature engineering based on artificial experience; on the other hand, the transaction vector is restored to the transaction dictionary code, and the transaction dictionary number with the highest probability is selected as the prediction result of the next transaction, so as to discover the deep-level rules contained in the massive transaction behavior, thereby reducing the dependence on manual labeling information and improving the accuracy of transaction data prediction.

[0192] Figure 6 A flow chart of a model training method for transaction data vectorization and prediction provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the method includes:

[0193] 401. Obtain a transaction sequence of the user to be trained, and perform dictionary coding on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the user to be trained at a historical moment.

[0194] Exemplarily, the execution subject of this embodiment may be an electronic device, hereinafter referred to as the device. The device may directly obtain the transaction sequence of the user to be trained from a local database or from an external server, wherein the transaction sequence includes multiple transaction messages, i.e., transaction data representing the user to be trained at historical moments, such as transaction records and transaction logs at multiple consecutive historical moments. The device calls a preset transaction vector encoder, which includes a transaction dictionary mapping sublayer and a transaction vector embedding layer. Based on the transaction dictionary mapping sublayer, dictionary encoding is performed on each transaction message to obtain a dictionary code corresponding to each transaction message, so as to correspond to each original transaction data of the user to be trained.

[0195] 402. Perform high-dimensional mapping processing on each dictionary code to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each dictionary code.

[0196] Exemplarily, based on the transaction vector embedding sublayer, each dictionary code obtained, such as each string of numbers, is projected into a high-dimensional space to generate a vector, and a transaction vector corresponding to each dictionary code is obtained to achieve a universal representation of each transaction data of the user to be trained, and then a vector set is obtained for processing.

[0197] 403. Based on the initial model, the vector set is processed to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation between each transaction vector.

[0198] Exemplarily, the device calls a preset initial model, and based on the self-attention sublayer of the initial model, performs self-attention processing on each transaction vector in the obtained vector set, and identifies the correlation between each transaction vector in the vector set through the processing of the residual connection and normalization sublayer, the forward connection layer, and the residual connection and normalization sublayer. The output result is expressed in matrix form, and the output matrix corresponding to the vector set is obtained.

[0199] 404. According to the output matrix, the initial model is trained to obtain a self-attention model; wherein the self-attention model is used to process the transaction sequence of the target user in the aforementioned transaction data vectorization and prediction method to obtain the transaction data prediction result of the target user; the transaction data prediction result represents the transaction status of the target user in the predicted future time period.

[0200] Exemplarily, the device trains the initial model according to the obtained output matrix, such as adjusting model parameters, etc., to obtain a self-attention model. The self-attention model can process the transaction sequence of each target user to obtain prediction data, i.e., transaction data prediction results, and then predict the transaction status of the target user in the future time period, such as whether the target user has transactions at multiple future moments in the future time period, or whether the target user's transaction items increase in the future time period, or the transaction data and transaction records generated when the target user trades in the future time period.

[0201] In this embodiment, a model training method for transaction data vectorization and prediction is provided, in which each transaction data of the user to be trained is processed into a transaction vector through dictionary encoding, and is input into the initial model to process all the transaction vectors to obtain an output matrix to discover the correlation between the transaction vectors, and the initial model is trained through the output matrix to obtain a self-attention model to predict the transaction status of the target user in the future time period; further, through the processing of the self-attention model, the correlation between the transaction vectors is discovered to perform transaction data prediction, which can improve the accuracy of transaction data prediction.

[0202] Figure 7 A flow chart of another model training method for transaction data vectorization and prediction provided in an embodiment of the present application is shown as follows: Figure 7 As shown, the method includes:

[0203] 501. Obtain a transaction sequence of a user to be trained, and perform dictionary coding on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents transaction data of the user to be trained at a historical moment.

[0204] Exemplarily, this step may refer to step 401 and will not be described in detail here.

[0205] In one example, step 501 includes the following steps:

[0206] The first step of step 501 is to discretize each message field in the transaction message to obtain a discretized value corresponding to each message field; wherein the message field represents a transaction field in the transaction data.

[0207] The second step of step 501 is to perform segmentation mapping processing on the discretized values ​​to obtain field values ​​corresponding to the message fields.

[0208] The third step of step 501 is to concatenate the values ​​of each field to obtain the dictionary code corresponding to the transaction message.

[0209] Exemplarily, each transaction message includes multiple message fields, such as transaction amount, time, date, card attributes, response code, merchant type, etc. Based on the transaction dictionary mapping layer, the continuous space of each field such as transaction amount, time, date, card attributes, response code, merchant type, transaction area, transaction type, etc. in the transaction message is discretized and mapped into a discrete value through interval mapping. For the i-th field in the transaction message, its corresponding value space is segmented and mapped to obtain the corresponding field value. The field values ​​in the transaction message are spliced ​​to obtain the dictionary code of each transaction message. The mapping logic of different transaction message fields is determined according to business understanding.

[0210] 502. Perform high-dimensional mapping processing on each dictionary code to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each dictionary code.

[0211] Exemplarily, this step may refer to step 402 and will not be described in detail here.

[0212] In one example, step 502 includes the following steps:

[0213] The first step of step 502 is to perform one-hot numbering processing on the dictionary code to obtain a one-hot number corresponding to the transaction message.

[0214] The second step of step 502 is to perform high-dimensional mapping processing on the unique hot number to obtain a transaction vector corresponding to the transaction message.

[0215] For example, based on the transaction vector embedding layer, the transaction dictionary encoding sequence obtained by the transaction dictionary mapping sublayer is numbered one-hot and converted into a transaction vector. First, the space size of the transaction dictionary encoding Trans_vec is Assign each code to a value between 0 and The unique serial number of the transaction dictionary is called the "one hot" number. Secondly, the "one hot" number of each transaction dictionary is mapped to a vector space with dimension dmodel to facilitate subsequent processing by the deep self-attention decoder. This process is called "vector embedding". The specific value of the transaction vector can be obtained through model learning.

[0216] 503. Based on at least one self-attention mechanism of the self-attention model, determine a query matrix, a key matrix and a value matrix corresponding to the self-attention mechanism; wherein the query matrix includes each transaction vector currently being processed in the vector set; the key matrix includes all transaction vectors in the vector set; and the value matrix includes a value vector representing each transaction vector itself.

[0217] Exemplarily, the initial model deployed by the device is an autoregressive decoder based on a deep self-attention neural network. Based on the self-attention sublayer, in order to more effectively expand the model's ability to pay attention to different attributes of transaction vectors, a multi-head self-attention mechanism is deployed. For each self-attention mechanism, according to each transaction vector in the vector set, the query matrix corresponding to the transaction vector currently being processed under the self-attention mechanism, the key matrix including all transaction vectors in the vector set, and the value matrix including the value vector representing each transaction vector itself are determined.

[0218] 504. Based on the respective attention mechanisms, self-attention processing is performed on the query matrix, key matrix, and value matrix corresponding to the respective attention mechanisms to obtain an output matrix corresponding to the vector set.

[0219] Exemplarily, through each self-attention mechanism, self-attention processing is performed on the query matrix, key matrix and value matrix corresponding to the self-attention mechanism. For example, the query matrix and key matrix can be processed by the self-attention mechanism to obtain the correlation between different transaction vectors, and then the respective value vectors in the value matrix are used to determine the final representation of the current transaction vector, thereby obtaining the output matrix corresponding to all transaction vectors corresponding to the self-attention mechanism.

[0220] In one example, step 504 includes the following steps:

[0221] The first step of step 504 is to determine the weight matrix corresponding to the self-attention mechanism according to the query matrix and the key matrix corresponding to the self-attention mechanism; wherein the weight matrix includes the weight coefficient corresponding to each transaction vector.

[0222] The second step of step 504 is to perform weighted summation processing on each value vector in the value matrix according to the weight matrix to obtain a self-attention matrix corresponding to the self-attention mechanism.

[0223] The second step of step 504 is to concatenate the respective attention matrices to obtain an output matrix corresponding to the vector set.

[0224] Exemplarily, through each self-attention mechanism, the query matrix and key matrix corresponding to the self-attention mechanism are self-attention weighted, and each transaction vector, that is, the weight coefficient corresponding to each currently processed transaction vector is obtained, and then the weight coefficients corresponding to all transaction vectors are combined to obtain the weight matrix corresponding to the self-attention mechanism. Based on each self-attention mechanism, the weight coefficients of the weight matrix and the value vectors in the value matrix are weighted and summed to obtain the self-attention matrix corresponding to all transaction vectors under the self-attention mechanism. Through the splicing operation, the self-attention matrices corresponding to each self-attention mechanism are tightly spliced ​​to obtain the output matrix. Then, in this way, the attention mechanism can dynamically adjust the contribution of each transaction vector in the final representation according to the correlation between the transaction vectors, so as to capture richer context information.

[0225] In one example, the first step of step 504 includes:

[0226] Step 1: Determine a similarity matrix between the query matrix and the key matrix; wherein the similarity matrix represents the correlation between each transaction vector and other transaction vectors except the transaction vector.

[0227] Step 2: Scaling the similarity matrix to obtain a scaled similarity matrix.

[0228] Step 3: Normalize the scaled similarity matrix to obtain the weight matrix corresponding to the self-attention mechanism.

[0229] Specifically, through each self-attention mechanism, the query matrix and key matrix corresponding to the self-attention mechanism are similarity calculated, and the similarity between each transaction vector being processed and the remaining transaction vectors except the transaction vector, that is, the correlation, is calculated, and then the similarity matrix between the query matrix and the key matrix is ​​combined; since the dimensions of the query matrix and the key matrix may be large, they need to be scaled, which helps to avoid the problem of gradient disappearance or gradient explosion when calculating the Softmax function, and then the scaled similarity matrix is ​​obtained; through the normalization algorithm of the self-attention mechanism, such as the Softmax function, the scaled similarity matrix is ​​normalized to obtain the weight matrix corresponding to the self-attention mechanism. It is worth adding that based on the above process, after obtaining the weight matrix, the self-attention matrix corresponding to the i-th self-attention mechanism is obtained through each self-attention mechanism.

[0230] 505. Perform encoding and restoration processing on the output matrix to obtain a transaction prediction result of the user to be trained; wherein the transaction prediction result represents the transaction situation of the user to be trained within a preset time period of the prediction.

[0231] Exemplarily, the device calls a preset transaction generator, and based on the transaction vector restoration sublayer of the transaction generator, encodes and restores the obtained output matrix, that is, restores the output matrix into corresponding multiple vectors, and based on the transaction dictionary mapping sublayer of the transaction generator, maps the restored multiple vectors to obtain prediction data, that is, transaction data prediction results, and then predicts the transaction situation of the user to be trained within a preset time period, such as whether the user to be trained has transactions at multiple preset time moments within the preset time period, or whether the transaction items of the user to be trained increase within the preset time period, or the transaction data and transaction records generated when the user to be trained trades within the preset time period.

[0232] In one example, step 505 includes the following steps:

[0233] Step 1: Perform dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix; wherein the predicted transaction sequence includes predicted dictionary codes at M preset moments within a preset time period; and the predicted dictionary codes represent the transaction data generated at the predicted preset moments.

[0234] Step 2: Perform field restoration and splicing processing on each prediction dictionary code to obtain a transaction data prediction result; wherein the transaction data prediction result includes predicted transaction messages at M preset moments; the predicted transaction message represents the transaction status of the user to be trained at the predicted preset moment.

[0235] Exemplarily, based on the transaction vector restoration sublayer, the output matrix of the autoregressive decoder is restored through a linear layer to the predicted dictionary code at M preset moments in the preset time period, that is, the predicted transaction data generated at each preset moment, and then the predicted transaction sequence is obtained. Based on the transaction dictionary mapping sublayer, each predicted dictionary code is restored to the field to obtain the corresponding predicted transaction message, that is, the predicted transaction status of the target user at each preset moment, and all predicted transaction messages are spliced ​​to obtain the transaction data prediction result.

[0236] In one example, step 1 of step 505 includes the following steps:

[0237] The first step is to perform dictionary coding restoration processing on the j-th output matrix to obtain the j-th prediction dictionary code in the predicted transaction sequence.

[0238] The second step is to generate a new transaction vector corresponding to the j-th prediction dictionary code; and add the new transaction vector corresponding to the j-th prediction dictionary code to the vector set.

[0239] The third step is to process the vector set based on the self-attention model to obtain the j+1th output matrix.

[0240] The fourth step is to perform dictionary coding restoration processing on the j+1th output matrix to obtain the j+1th predicted dictionary code in the predicted transaction sequence.

[0241] Exemplarily, based on the transaction vector restoration sublayer, for the first output matrix, a dictionary coding restoration process is performed through a linear layer to restore the predicted dictionary coding at the first future moment in a preset time period, and the first predicted dictionary coding is input into the transaction vector encoder to perform vector mapping processing on the first predicted dictionary coding to obtain a new transaction vector of the first predicted dictionary coding, that is, the first new transaction vector, and the first new transaction vector is input into the vector set, that is, into the vector set initially obtained to obtain a new vector set, and the new vector set is input into the self-attention in the autoregressive decoder. In the force model, processing is performed, and the processing process is consistent with the aforementioned processing process to obtain the second output matrix. Based on the transaction vector restoration sublayer, for the second output matrix, dictionary coding restoration processing is performed through a linear layer to restore it to the predicted dictionary coding at the second future moment in the preset time period. The above process is repeated to obtain the jth output matrix. Through the dictionary coding restoration processing, vector mapping, adding vector sets, and self-attention model processing in the aforementioned process, the j+1th output matrix is ​​obtained, and then the dictionary coding restoration processing is performed to obtain the j+1th predicted dictionary coding to obtain the final predicted transaction sequence.

[0242] In one example, the first step includes:

[0243] Step 1: Perform dictionary coding restoration processing on the j-th output matrix to obtain at least one restored dictionary code corresponding to the j-th output matrix; wherein the restored dictionary code has a prediction probability, and the prediction probability is the generation probability of the predicted transaction data.

[0244] Step 2: Determine the restored dictionary code with the highest prediction probability from at least one restored dictionary code corresponding to the j-th output matrix as the j-th predicted dictionary code in the predicted transaction sequence.

[0245] Specifically, for the j-th output matrix, based on the transaction vector restoration sublayer, the j-th output matrix is ​​subjected to dictionary coding restoration processing to obtain multiple restored dictionary codes corresponding to the j-th output matrix, each restored dictionary code having a prediction probability obtained by self-attention model processing, that is, the generation probability of the predicted transaction data corresponding to the restored dictionary code; from all the restored dictionary codes, the restored dictionary code with the largest prediction probability is selected, and the restored dictionary code is determined as the j-th predicted dictionary code in the predicted transaction sequence, that is, the predicted result of the next transaction that will occur at the preset moment.

[0246] In one example, step 2 includes the following steps:

[0247] The first step is to perform field restoration processing on the prediction dictionary code to obtain at least one prediction message field corresponding to the prediction dictionary code; wherein the prediction message field represents a transaction field in the transaction data generated at a preset prediction moment.

[0248] The second step is to perform mapping processing on at least one predicted message field corresponding to the predicted dictionary code to obtain at least one predicted message value corresponding to the predicted dictionary code.

[0249] The third step is to perform concatenation processing on at least one predicted message value corresponding to the predicted dictionary code to obtain a predicted transaction message corresponding to the predicted dictionary code.

[0250] For example, based on the transaction dictionary mapping sublayer, each prediction dictionary code is restored to the real multiple prediction message fields corresponding to each prediction dictionary code, such as transaction amount, transaction type and other information, through the mapping space Domain_veci2=f -1 (Domain_veci2), map each predicted message field to the actual transaction message value, that is, the predicted message value, concatenate all the predicted message values ​​corresponding to each predicted dictionary code, and obtain the predicted transaction message corresponding to each predicted dictionary code.

[0251] 506. The initial model is trained according to the transaction prediction results and actual transaction results of the user to be trained to obtain a self-attention model; wherein the actual transaction results represent the transaction status of the user to be trained within the actual preset time period.

[0252] Exemplarily, the device compares the predicted transaction results of the user to be trained with the actual transaction results, and compares the predicted transaction status of the user to be trained within a preset time period with the actual transaction status of the user to be trained within a preset time period. Based on the degree of deviation between the two, the model parameters can be optimized and adjusted to obtain a self-attention model.

[0253] In this embodiment, based on the above embodiment, the original transaction is dictionary encoded and then projected into a vector in a high-dimensional space to achieve a universal representation of the transaction, and there is no need to design feature engineering based on artificial experience; on the other hand, the transaction vector is restored to the transaction dictionary code, and the transaction dictionary number with the highest probability is selected as the prediction result of the next transaction, so as to discover the deep-level rules contained in the massive transaction behavior, thereby reducing the dependence on manually labeled information and obtaining a more accurate prediction model to improve the accuracy of transaction data prediction.

[0254] Figure 8 A schematic diagram of the structure of a transaction data vectorization and prediction device provided in an embodiment of the present application, such as Figure 8 As shown, the device comprises:

[0255] The encoding module 601 is used to obtain the transaction sequence of the target user, and perform dictionary encoding processing on each transaction message in the transaction sequence to obtain the dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the target user at a historical moment;

[0256] A mapping module 602 is used to perform high-dimensional mapping processing on each dictionary code to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each dictionary code;

[0257] The processing module 603 is used to process the vector set based on the self-attention model to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation relationship between each transaction vector;

[0258] The restoration module 604 is used to perform encoding restoration processing on the output matrix to obtain the transaction data prediction result of the target user; wherein the transaction data prediction result represents the transaction situation of the target user in the predicted future time period.

[0259] In one possible implementation, the processing module 603 is specifically used to: determine the query matrix, key matrix and value matrix corresponding to the self-attention mechanism based on at least one self-attention mechanism of the self-attention model; wherein the query matrix includes each transaction vector currently being processed in the vector set; the key matrix includes all transaction vectors in the vector set; the value matrix includes the value vector representing each transaction vector itself; based on the respective attention mechanisms, perform self-attention processing on the query matrix, key matrix and value matrix corresponding to the respective attention mechanisms to obtain an output matrix corresponding to the vector set.

[0260] In one possible implementation, the processing module 603 is specifically used to: determine a weight matrix corresponding to the self-attention mechanism based on a query matrix and a key matrix corresponding to the self-attention mechanism; wherein the weight matrix includes a weight coefficient corresponding to each transaction vector; perform weighted summation processing on each value vector in the value matrix according to the weight matrix to obtain a self-attention matrix corresponding to the self-attention mechanism; and perform splicing processing on each attention matrix to obtain an output matrix corresponding to the vector set.

[0261] In one possible implementation, the processing module 603 is specifically used to: determine a similarity matrix between a query matrix and a key matrix; wherein the similarity matrix represents the correlation between each transaction vector and the remaining transaction vectors except the transaction vector; scale the similarity matrix to obtain a scaled similarity matrix; and normalize the scaled similarity matrix to obtain a weight matrix corresponding to the self-attention mechanism.

[0262] In a possible implementation, the restoration module 604 is specifically used to: perform dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix; wherein the predicted transaction sequence includes predicted dictionary codes at N future moments within a preset time period; the predicted dictionary codes represent the transaction data generated at the predicted future moments; perform field restoration and splicing processing on each predicted dictionary code to obtain a transaction data prediction result; wherein the transaction data prediction result includes predicted transaction messages at N future moments; the predicted transaction messages represent the transaction status of the target user at the predicted future moments.

[0263] In a possible implementation, the restoration module 604 is specifically used to: perform dictionary coding restoration processing on the i-th output matrix to obtain the i-th predicted dictionary code in the predicted transaction sequence;

[0264] Generate a new transaction vector corresponding to the i-th prediction dictionary code; add the new transaction vector corresponding to the i-th prediction dictionary code to the vector set; process the vector set based on the self-attention model to obtain the i+1-th output matrix; perform dictionary code restoration on the i+1-th output matrix to obtain the i+1-th prediction dictionary code in the predicted transaction sequence.

[0265] In a possible implementation, the restoration module 604 is specifically used to: perform dictionary coding restoration processing on the i-th output matrix to obtain at least one restored dictionary code corresponding to the i-th output matrix; wherein the restored dictionary code has a prediction probability, and the prediction probability is the generation probability of the predicted transaction data; determine the restoration dictionary code with the highest prediction probability from the at least one restored dictionary code corresponding to the i-th output matrix as the i-th predicted dictionary code in the predicted transaction sequence.

[0266] In a possible implementation, the restoration module 604 is further specifically used to: perform field restoration processing on the prediction dictionary code to obtain at least one prediction message field corresponding to the prediction dictionary code; wherein the prediction message field represents a transaction field in the transaction data generated at a predicted future moment; perform mapping processing on at least one prediction message field corresponding to the prediction dictionary code to obtain at least one prediction message value corresponding to the prediction dictionary code; perform splicing processing on at least one prediction message value corresponding to the prediction dictionary code to obtain a predicted transaction message corresponding to the prediction dictionary code.

[0267] In a possible implementation, the encoding module 601 is specifically used to: discretize each message field in the transaction message to obtain a discretized value corresponding to each message field; wherein the message field represents a transaction field in the transaction data; segment and map the discretized value to obtain a field value corresponding to the message field; and concatenate the field values ​​to obtain a dictionary code corresponding to the transaction message.

[0268] In a possible implementation, the mapping module 602 is specifically used to: perform one-hot numbering processing on the dictionary code to obtain a one-hot number corresponding to the transaction message; and perform high-dimensional mapping processing on the one-hot number to obtain a transaction vector corresponding to the transaction message.

[0269] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0270] Fig. 9 A schematic diagram of the structure of a model training device for transaction data vectorization and prediction provided in an embodiment of the present application, such as Fig. 9 As shown, the device comprises:

[0271] The encoding module 701 is used to obtain the transaction sequence of the user to be trained, and perform dictionary encoding processing on each transaction message in the transaction sequence to obtain the dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the user to be trained at a historical moment;

[0272] A mapping module 702 is used to perform high-dimensional mapping processing on each dictionary code to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each dictionary code;

[0273] The processing module 703 is used to process the vector set based on the initial model to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation between each transaction vector;

[0274] The training module 704 is used to train the initial model according to the output matrix to obtain a self-attention model; wherein the self-attention model is used to process the transaction sequence of the target user in the transaction data vectorization and prediction device to obtain the transaction data prediction result of the target user; the transaction data prediction result represents the transaction status of the target user in the predicted future time period.

[0275] In one possible implementation, the processing module 703 is specifically used to: determine the query matrix, key matrix and value matrix corresponding to the self-attention mechanism based on at least one self-attention mechanism of the self-attention model; wherein the query matrix includes each transaction vector currently being processed in the vector set; the key matrix includes all transaction vectors in the vector set; the value matrix includes the value vector representing each transaction vector itself; based on the respective attention mechanisms, perform self-attention processing on the query matrix, key matrix and value matrix corresponding to the respective attention mechanisms to obtain an output matrix corresponding to the vector set.

[0276] In one possible implementation, the processing module 703 is specifically used to: determine a weight matrix corresponding to the self-attention mechanism based on a query matrix and a key matrix corresponding to the self-attention mechanism; wherein the weight matrix includes a weight coefficient corresponding to each transaction vector; according to the weight matrix, perform weighted summation processing on each value vector in the value matrix to obtain a self-attention matrix corresponding to the self-attention mechanism; and concatenate each attention matrix to obtain an output matrix corresponding to the vector set.

[0277] In one possible implementation, the processing module 703 is specifically used to: determine a similarity matrix between a query matrix and a key matrix; wherein the similarity matrix represents the correlation between each transaction vector and the remaining transaction vectors except the transaction vector; scale the similarity matrix to obtain a scaled similarity matrix; and normalize the scaled similarity matrix to obtain a weight matrix corresponding to the self-attention mechanism.

[0278] In one possible implementation, the training module 704 is specifically used to: perform encoding and restoration processing on the output matrix to obtain transaction prediction results of the user to be trained; wherein the transaction prediction results represent the transaction status of the user to be trained within the predicted preset time period; perform training processing on the initial model according to the transaction prediction results and actual transaction results of the user to be trained to obtain a self-attention model; wherein the actual transaction results represent the transaction status of the user to be trained within the actual preset time period.

[0279] In a possible implementation, the training module 704 is specifically used to: perform dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix; wherein the predicted transaction sequence includes predicted dictionary codes at M preset moments within a preset time period; the predicted dictionary codes represent the transaction data generated at the predicted preset moments; perform field restoration and splicing processing on each predicted dictionary code to obtain a transaction data prediction result; wherein the transaction data prediction result includes predicted transaction messages at M preset moments; the predicted transaction messages represent the transaction status of the user to be trained at the predicted preset moments.

[0280] In a possible implementation, the training module 704 is specifically used to: perform dictionary coding restoration processing on the j-th output matrix to obtain the j-th predicted dictionary code in the predicted transaction sequence;

[0281] Generate a new transaction vector corresponding to the j-th prediction dictionary code; add the new transaction vector corresponding to the j-th prediction dictionary code to the vector set; process the vector set based on the self-attention model to obtain the j+1-th output matrix; perform dictionary code restoration on the j+1-th output matrix to obtain the j+1-th prediction dictionary code in the predicted transaction sequence.

[0282] In a possible implementation, the training module 704 is specifically used to: perform dictionary coding restoration processing on the j-th output matrix to obtain at least one restored dictionary code corresponding to the j-th output matrix; wherein the restored dictionary code has a prediction probability, and the prediction probability is the generation probability of the predicted transaction data; and determine the restored dictionary code with the highest prediction probability from the at least one restored dictionary code corresponding to the j-th output matrix as the j-th predicted dictionary code in the predicted transaction sequence.

[0283] In a possible implementation, the training module 704 is further specifically used to: perform field restoration processing on the prediction dictionary code to obtain at least one prediction message field corresponding to the prediction dictionary code; wherein the prediction message field represents a transaction field in the transaction data generated at a preset time of prediction; perform mapping processing on at least one prediction message field corresponding to the prediction dictionary code to obtain at least one prediction message value corresponding to the prediction dictionary code; perform splicing processing on at least one prediction message value corresponding to the prediction dictionary code to obtain a predicted transaction message corresponding to the prediction dictionary code.

[0284] In a possible implementation, the encoding module 701 is specifically used to: discretize each message field in the transaction message to obtain a discretized value corresponding to each message field; wherein the message field represents a transaction field in the transaction data; segment and map the discretized value to obtain a field value corresponding to the message field; and concatenate the field values ​​to obtain a dictionary code corresponding to the transaction message.

[0285] In a possible implementation, the mapping module 702 is specifically used to: perform one-hot numbering processing on the dictionary code to obtain a one-hot number corresponding to the transaction message; and perform high-dimensional mapping processing on the one-hot number to obtain a transaction vector corresponding to the transaction message.

[0286] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0287] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig.10 As shown, the electronic device includes: a memory 801 and a processor 802; the memory 801 is a memory for storing instructions executable by the processor 802.

[0288] The processor 802 is configured to execute the method provided in the above embodiment.

[0289] The electronic device further includes a receiver 803 and a transmitter 804. The receiver 803 is used to receive instructions and data sent by other devices, and the transmitter 804 is used to send instructions and data to external devices.

[0290] The specific implementation process of the processor can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0291] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0292] The memory may include a high-speed memory (Random Access Memory, referred to as RAM), and may also include a non-volatile memory (Non-volatile Memory, referred to as NVM), such as at least one disk memory.

[0293] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the processing method in the above embodiment.

[0294] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed on a computer, the computer executes the technical solution of the processing method of the above embodiment.

[0295] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0296] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in a dedicated integrated circuit. Of course, the processor and the readable storage medium can also exist in a device as discrete components.

[0297] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the processing method in the above embodiment can be implemented.

[0298] Those skilled in the art will appreciate that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed.

[0299] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A transaction data vectorization and prediction method, characterized in that: include: Acquire a transaction sequence of a target user, and perform dictionary coding on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the target user at a historical moment; Performing high-dimensional mapping processing on each of the dictionary codes to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each of the dictionary codes; Based on the self-attention model, the vector set is processed to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation between each transaction vector; The output matrix is ​​coded and restored to obtain a transaction data prediction result of the target user; wherein the transaction data prediction result represents the transaction situation of the target user in a predicted future time period.

2. The method according to claim 1, characterized in that Based on the self-attention model, the vector set is processed to obtain an output matrix corresponding to the vector set, including: Based on at least one self-attention mechanism of the self-attention model, determine a query matrix, a key matrix and a value matrix corresponding to the self-attention mechanism; wherein the query matrix includes each transaction vector currently being processed in the vector set; the key matrix includes all transaction vectors in the vector set; and the value matrix includes a value vector representing each transaction vector itself; Based on each of the self-attention mechanisms, self-attention processing is performed on the query matrix, key matrix and value matrix corresponding to each of the self-attention mechanisms to obtain an output matrix corresponding to the vector set.

3. The method according to claim 2, characterized in that Based on each of the self-attention mechanisms, self-attention processing is performed on the query matrix, key matrix, and value matrix corresponding to each of the self-attention mechanisms to obtain an output matrix corresponding to the vector set, including: Determine a weight matrix corresponding to the self-attention mechanism according to the query matrix and the key matrix corresponding to the self-attention mechanism; wherein the weight matrix includes a weight coefficient corresponding to each of the transaction vectors; According to the weight matrix, performing weighted summation processing on each value vector in the value matrix to obtain a self-attention matrix corresponding to the self-attention mechanism; Each of the self-attention matrices is concatenated to obtain an output matrix corresponding to the vector set.

4. The method according to claim 3, characterized in that Determining a weight matrix corresponding to the self-attention mechanism according to a query matrix and a key matrix corresponding to the self-attention mechanism includes: Determine a similarity matrix between the query matrix and the key matrix; wherein the similarity matrix represents the correlation between each transaction vector and other transaction vectors except the transaction vector; Scaling the similarity matrix to obtain a scaled similarity matrix; The scaled similarity matrix is ​​normalized to obtain a weight matrix corresponding to the self-attention mechanism.

5. The method according to claim 1, characterized in that The output matrix is ​​encoded and restored to obtain the transaction data prediction result of the target user, including: Performing dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix; wherein the predicted transaction sequence includes predicted dictionary codes at N future moments within a preset time period; the predicted dictionary codes represent transaction data generated at the predicted future moments; The prediction dictionary codes are subjected to field restoration and splicing processing to obtain the transaction data prediction result; wherein the transaction data prediction result includes predicted transaction messages at N future moments; the predicted transaction messages represent the transaction status of the target user at the predicted future moment.

6. The method according to claim 5, characterized in that Performing dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix includes: Performing dictionary coding restoration processing on the i-th output matrix to obtain the i-th prediction dictionary code in the predicted transaction sequence; Generate a new transaction vector corresponding to the i-th prediction dictionary code; and add the new transaction vector corresponding to the i-th prediction dictionary code to the vector set; Based on the self-attention model, the vector set is processed to obtain an i+1th output matrix; The i+1th output matrix is ​​subjected to dictionary coding restoration processing to obtain the i+1th predicted dictionary code in the predicted transaction sequence.

7. The method according to claim 6, characterized in that Performing dictionary coding restoration processing on the i-th output matrix to obtain the i-th prediction dictionary code in the predicted transaction sequence includes: Performing dictionary coding restoration processing on the i-th output matrix to obtain at least one restored dictionary code corresponding to the i-th output matrix; wherein the restored dictionary code has a prediction probability, and the prediction probability is a generation probability of the predicted transaction data; The restored dictionary code with the highest prediction probability is determined from at least one restored dictionary code corresponding to the i-th output matrix as the i-th predicted dictionary code in the predicted transaction sequence.

8. The method according to claim 5, characterized in that Performing field restoration and splicing processing on each of the prediction dictionary codes to obtain the transaction data prediction result includes: Performing field restoration processing on the prediction dictionary code to obtain at least one prediction message field corresponding to the prediction dictionary code; wherein the prediction message field represents a transaction field in the transaction data generated at a predicted future moment; Performing mapping processing on at least one predicted message field corresponding to the predicted dictionary code to obtain at least one predicted message value corresponding to the predicted dictionary code; At least one predicted message value corresponding to the predicted dictionary code is concatenated to obtain a predicted transaction message corresponding to the predicted dictionary code.

9. The method according to claim 1, characterized in that: Performing dictionary coding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message includes: Discretization processing is performed on each message field in the transaction message to obtain a discretized value corresponding to each message field; wherein the message field represents a transaction field in the transaction data; Perform segmentation and mapping processing on the discretized value to obtain a field value corresponding to the message field; The values ​​of each field are concatenated to obtain a dictionary code corresponding to the transaction message.

10. The method according to any one of claims 1 to 9, characterized in that A high-dimensional mapping process is performed on each of the dictionary codes to obtain a vector set, including: Performing one-hot numbering processing on the dictionary code to obtain a one-hot number corresponding to the transaction message; A high-dimensional mapping process is performed on the unique hot number to obtain a transaction vector corresponding to the transaction message.

11. A model training method for transaction data vectorization and prediction, characterized in that: include: Acquire a transaction sequence of the user to be trained, and perform dictionary coding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the user to be trained at a historical moment; Performing high-dimensional mapping processing on each of the dictionary codes to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each of the dictionary codes; Based on the initial model, the vector set is processed to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation relationship between each transaction vector; The initial model is trained according to the output matrix to obtain a self-attention model; wherein the self-attention model is used to process the transaction sequence of the target user as described in any one of claims 1 to 10 to obtain the transaction data prediction result of the target user; the transaction data prediction result represents the transaction status of the target user in the predicted future time period.

12. The method according to claim 11, characterized in that Based on the initial model, the vector set is processed to obtain an output matrix corresponding to the vector set, including: Based on at least one self-attention mechanism of the self-attention model, determine a query matrix, a key matrix and a value matrix corresponding to the self-attention mechanism; wherein the query matrix includes each transaction vector currently being processed in the vector set; the key matrix includes all transaction vectors in the vector set; and the value matrix includes a value vector representing each transaction vector itself; Based on each of the self-attention mechanisms, self-attention processing is performed on the query matrix, key matrix and value matrix corresponding to each of the self-attention mechanisms to obtain an output matrix corresponding to the vector set.

13. The method according to claim 12, characterized in that Based on each of the self-attention mechanisms, self-attention processing is performed on the query matrix, key matrix, and value matrix corresponding to each of the self-attention mechanisms to obtain an output matrix corresponding to the vector set, including: Determine a weight matrix corresponding to the self-attention mechanism according to the query matrix and the key matrix corresponding to the self-attention mechanism; wherein the weight matrix includes a weight coefficient corresponding to each of the transaction vectors; According to the weight matrix, performing weighted summation processing on each value vector in the value matrix to obtain a self-attention matrix corresponding to the self-attention mechanism; Each of the self-attention matrices is concatenated to obtain an output matrix corresponding to the vector set.

14. The method according to claim 13, characterized in that Determining a weight matrix corresponding to the self-attention mechanism according to a query matrix and a key matrix corresponding to the self-attention mechanism includes: Determine a similarity matrix between the query matrix and the key matrix; wherein the similarity matrix represents the correlation between each transaction vector and other transaction vectors except the transaction vector; Scaling the similarity matrix to obtain a scaled similarity matrix; The scaled similarity matrix is ​​normalized to obtain a weight matrix corresponding to the self-attention mechanism.

15. The method according to claim 11, characterized in that According to the output matrix, the initial model is trained to obtain a self-attention model, including: The output matrix is ​​encoded and restored to obtain a transaction prediction result of the user to be trained; wherein the transaction prediction result represents the transaction situation of the user to be trained within a preset time period of the prediction; The initial model is trained according to the transaction prediction results and actual transaction results of the user to be trained to obtain the self-attention model; wherein the actual transaction results represent the transaction status of the user to be trained within the actual preset time period.

16. The method according to claim 15, characterized in that The output matrix is ​​encoded and restored to obtain the transaction prediction result of the user to be trained, including: Performing dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix; wherein the predicted transaction sequence includes predicted dictionary codes at M preset moments within a preset time period; the predicted dictionary codes represent transaction data generated at the predicted preset moments; Field restoration and splicing processing is performed on each of the prediction dictionary codes to obtain the transaction data prediction result; wherein the transaction data prediction result includes predicted transaction messages at M preset moments; the predicted transaction message represents the transaction status of the user to be trained at the predicted preset moment.

17. The method according to claim 16, characterized in that Performing dictionary coding restoration processing on the output matrix to obtain a predicted transaction sequence corresponding to the output matrix includes: Performing dictionary coding restoration processing on the j-th output matrix to obtain the j-th prediction dictionary code in the predicted transaction sequence; Generate a new transaction vector corresponding to the j-th prediction dictionary code; and add the new transaction vector corresponding to the j-th prediction dictionary code to the vector set; Based on the self-attention model, the vector set is processed to obtain the j+1th output matrix; The j+1th output matrix is ​​subjected to dictionary coding restoration processing to obtain the j+1th predicted dictionary code in the predicted transaction sequence.

18. The method according to claim 17, characterized in that Performing dictionary coding restoration processing on the j-th output matrix to obtain the j-th prediction dictionary code in the predicted transaction sequence includes: Performing dictionary coding restoration processing on the j-th output matrix to obtain at least one restored dictionary code corresponding to the j-th output matrix; wherein the restored dictionary code has a prediction probability, and the prediction probability is a generation probability of the predicted transaction data; The restored dictionary code with the highest prediction probability is determined from at least one restored dictionary code corresponding to the j-th output matrix as the j-th predicted dictionary code in the predicted transaction sequence.

19. The method according to claim 16, characterized in that Performing field restoration and splicing processing on each of the prediction dictionary codes to obtain the transaction data prediction result includes: Performing field restoration processing on the prediction dictionary code to obtain at least one prediction message field corresponding to the prediction dictionary code; wherein the prediction message field represents a transaction field in the transaction data generated at a preset time of prediction; Performing mapping processing on at least one predicted message field corresponding to the predicted dictionary code to obtain at least one predicted message value corresponding to the predicted dictionary code; At least one predicted message value corresponding to the predicted dictionary code is concatenated to obtain a predicted transaction message corresponding to the predicted dictionary code.

20. The method according to claim 11, characterized in that Performing dictionary coding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message includes: Discretization processing is performed on each message field in the transaction message to obtain a discretized value corresponding to each message field; wherein the message field represents a transaction field in the transaction data; Perform segmentation and mapping processing on the discretized value to obtain a field value corresponding to the message field; The values ​​of each field are concatenated to obtain a dictionary code corresponding to the transaction message.

21. The method according to any one of claims 11 to 20, characterized in that A high-dimensional mapping process is performed on each of the dictionary codes to obtain a vector set, including: Performing one-hot numbering processing on the dictionary code to obtain a one-hot number corresponding to the transaction message; A high-dimensional mapping process is performed on the unique hot number to obtain a transaction vector corresponding to the transaction message.

22. A transaction data vectorization and prediction device, characterized in that: include: An encoding module, used to obtain a transaction sequence of a target user, and perform dictionary encoding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the target user at a historical moment; A mapping module, used for performing high-dimensional mapping processing on each of the dictionary codes to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each of the dictionary codes; A processing module, configured to process the vector set based on a self-attention model to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation between each transaction vector; The restoration module is used to perform encoding restoration processing on the output matrix to obtain the transaction data prediction result of the target user; wherein the transaction data prediction result represents the transaction situation of the target user in the predicted future time period.

23. A model training device for transaction data vectorization and prediction, characterized in that: include: An encoding module, used to obtain a transaction sequence of a user to be trained, and to perform dictionary encoding processing on each transaction message in the transaction sequence to obtain a dictionary code corresponding to each transaction message; wherein the transaction message represents the transaction data of the user to be trained at a historical moment; A mapping module, used for performing high-dimensional mapping processing on each of the dictionary codes to obtain a vector set; wherein the vector set includes a transaction vector corresponding to each of the dictionary codes; A processing module, used to process the vector set based on the initial model to obtain an output matrix corresponding to the vector set; wherein the output matrix represents the correlation relationship between each transaction vector; A training module is used to train the initial model according to the output matrix to obtain a self-attention model; wherein the self-attention model is used to process the transaction sequence of the target user as described in claim 22 to obtain the transaction data prediction result of the target user; the transaction data prediction result represents the transaction status of the target user in the predicted future time period.

24. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 21.

25. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 21.

26. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 21 is implemented.

Images