A time series data processing method, apparatus, device, and program product

Abstract

The application discloses a kind of time series data processing method, device, equipment and program product, the method includes: for each transaction record in target transaction sequence, each transaction feature in record is respectively vectorized, and the feature vector sequence of transaction record is obtained;According to the feature vector sequence of each record and the occurrence time of each record, the target feature tensor of sequence is generated;The target feature tensor is input into feature encoder, and the integrated feature vector of record is obtained;Integrated feature vector of record is sequentially input into sequence decoder according to occurrence time, and the context memory vector of sequence at target time step is obtained, and context memory vector is used to execute classification task to sequence.The application can retain the independent attribute and association of each feature inside single transaction, realize the effective fusion of structured data and natural language text, improve the fine granularity and richness of transaction representation, and can meet the demand of deep understanding of transaction sequence.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a time-series data processing method, apparatus, device and program product. Background Technology

[0002] In recent years, large model technology has powerful capabilities for representing sequence data. To adapt to the structured characteristics of financial transaction sequences, related technical solutions have drawn on the token sequence processing logic of language large models. Each transaction is mapped to a single token, forming a transaction token sequence consistent with the language large model. Then, through operations such as embedding and self-attention, the transaction sequence is represented and modeled.

[0003] However, the above-mentioned schemes fail to fully integrate the semantics of structured transaction features with unstructured natural language text, resulting in insufficient fine-grainedness and richness of transaction representations, making it difficult to meet the needs for in-depth understanding of transaction sequences.

[0004] Based on this, this application provides a time-series data processing method. Summary of the Invention

[0005] This application provides a time-series data processing method, apparatus, device, computer-readable storage medium, and computer program product, which can improve the fineness and richness of transaction representation and meet the needs for in-depth understanding of transaction sequences.

[0006] In a first aspect, embodiments of this application provide a time-series data processing method, the method comprising: For each transaction record in the target transaction sequence, each transaction feature in the transaction record is vectorized to obtain a feature vector sequence for each transaction record, wherein the feature vector sequence includes the feature vectors corresponding to each transaction feature. Based on the feature vector sequence of each transaction record and the transaction occurrence time of each transaction record, generate the target feature tensor of the target transaction sequence; The target feature tensor is input into the feature encoder, and the feature encoder encodes the correlation between each feature vector in the feature vector sequence of each transaction record to obtain the comprehensive feature vector of each transaction record. The comprehensive feature vector of each transaction record is sequentially input into the sequence decoder according to the transaction occurrence time of each transaction record. The sequence decoder analyzes the temporal correlation between multiple comprehensive feature vectors to obtain the context memory vector of the target transaction sequence at the target time step based on the temporal correlation. The context memory vector is used to perform a classification task on the target transaction sequence.

[0007] Secondly, embodiments of this application provide a time-series data processing apparatus, the apparatus comprising: The transaction feature processing module is used to vectorize each transaction feature in each transaction record in the target transaction sequence to obtain a feature vector sequence for each transaction record, wherein the feature vector sequence includes the feature vectors corresponding to each transaction feature. The feature tensor generation module is used to generate the target feature tensor of the target transaction sequence based on the feature vector sequence of each transaction record and the transaction occurrence time of each transaction record. The feature vector encoding module is used to input the target feature tensor into the feature encoder, and to encode the correlation between each feature vector in the feature vector sequence of each transaction record to obtain the comprehensive feature vector of each transaction record. The memory vector processing module is used to input the comprehensive feature vector of each transaction record into the sequence decoder in sequence according to the transaction occurrence time of each transaction record. The sequence decoder analyzes the temporal correlation between multiple comprehensive feature vectors to obtain the context memory vector of the target transaction sequence at the target time step based on the temporal correlation. The context memory vector is used to perform a classification task on the target transaction sequence.

[0008] Thirdly, embodiments of this application provide an electronic device, which includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements any of the possible implementations of the first aspect described above.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the method in any of the possible implementations of the first aspect described above.

[0010] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform a method as described in any of the possible implementations of the first aspect above.

[0011] In this embodiment, for each transaction record in the target transaction sequence, each transaction feature in the transaction record is vectorized to obtain a feature vector sequence for each transaction record. The feature vector sequence includes a sequence of feature vectors corresponding to each transaction feature. Based on the feature vector sequence of each transaction record and the transaction occurrence time of each transaction record, a target feature tensor of the target transaction sequence is generated. The target feature tensor is input into a feature encoder, which encodes the correlation between the feature vectors in the feature vector sequence of each transaction record to obtain a comprehensive feature vector for each transaction record. The comprehensive feature vector of each transaction record is input into a sequence decoder according to the transaction occurrence time of each transaction record. The sequence decoder analyzes the temporal correlation between multiple comprehensive feature vectors to obtain the context memory vector of the target transaction sequence at the target time step based on the temporal correlation. The context memory vector is used to perform a classification task on the target transaction sequence. Because all transaction features of each transaction record in the transaction sequence are vectorized, the independent attributes and relationships of each feature within a single transaction are fully preserved, achieving effective fusion of structured data and natural language text. Therefore, this application can process the feature tensor of the transaction sequence through a feature encoder to encode the relationship between feature vectors within a single transaction to mine the semantic logic within the transaction. Then, the sequence decoder analyzes the temporal relationship between the comprehensive feature vectors of multiple transactions and generates a context memory vector. With the hierarchical division of labor and cooperation between the feature encoder and the sequence decoder, the hierarchical processing significantly improves the fineness and richness of the transaction representation. At the same time, the context memory vector is applied to the transaction sequence classification task, effectively improving the understanding of transaction behavior and the accuracy of classification. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart illustrating a time-series data processing method provided in an embodiment of this application; Figure 2 A schematic diagram of a token matrix provided in an embodiment of this application; Figure 3 A schematic diagram of an encoding process provided in an embodiment of this application; Figure 4 A schematic diagram of an analysis process provided for an embodiment of this application; Figure 5 This application provides a schematic diagram of the overall process of time-series data processing. Figure 6 A schematic diagram of a training process provided in an embodiment of this application; Figure 7 A schematic diagram of the structure of a time-series data processing device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0015] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0016] Furthermore, the acquisition, storage, use, and processing of data in this application's technical solution all comply with relevant national laws and regulations.

[0017] As described in the background section, current methods for processing financial transaction sequences rely too heavily on large language models for vector representation, failing to fully explore and reflect the rich connotations of financial transaction data. Specific problems include, but are not limited to: loss of internal transaction feature information: treating the entire transaction as an indivisible atomic unit for prediction ignores the rich logical connections and dependencies between features within a single transaction. Flattening of transaction sequence representation: treating each transaction as a flat, independent "vocabulary" set for processing. Insufficient integration of structured data and unstructured natural language: processing all transaction field features as category features fails to effectively address the deep integration problem between structured transaction features and unstructured natural language text, severing its inherent linguistic logic and semantic information.

[0018] Therefore, to address the problems of the prior art, embodiments of this application provide a time-series data processing method, apparatus, device, computer-readable storage medium, and computer program product. This time-series data processing method can be applied to any scenario where time-series data processing is required.

[0019] The execution subject of this application embodiment includes an electronic device capable of executing a timing data processing method.

[0020] The timing data processing method provided in the embodiments of this application will be introduced below.

[0021] Figure 1 A flowchart illustrating a time-series data processing method provided in an embodiment of this application is shown. Figure 1 As shown, the time-series data processing method provided in this application includes the following steps: S110: For each transaction record in the target transaction sequence, vectorize each transaction feature in the transaction record to obtain a feature vector sequence for each transaction record.

[0022] S120: Generate the target feature tensor of the target transaction sequence based on the feature vector sequence of each transaction record and the transaction occurrence time of each transaction record.

[0023] S130: Input the target feature tensor into the feature encoder. The feature encoder encodes the correlation between the feature vectors in the feature vector sequence of each transaction record to obtain the comprehensive feature vector of each transaction record.

[0024] S140: Input the comprehensive feature vector of each transaction record into the sequence decoder in sequence according to the transaction occurrence time of each transaction record. Analyze the temporal correlation between multiple comprehensive feature vectors through the sequence decoder to obtain the context memory vector of the target transaction sequence at the target time step based on the temporal correlation.

[0025] In this embodiment, for each transaction record in the target transaction sequence, each transaction feature in the transaction record is vectorized to obtain a feature vector sequence for each transaction record. The feature vector sequence includes a sequence of feature vectors corresponding to each transaction feature. Based on the feature vector sequence of each transaction record and the transaction occurrence time of each transaction record, a target feature tensor of the target transaction sequence is generated. The target feature tensor is input into a feature encoder, which encodes the correlation between the feature vectors in the feature vector sequence of each transaction record to obtain a comprehensive feature vector for each transaction record. The comprehensive feature vector of each transaction record is input into a sequence decoder according to the transaction occurrence time of each transaction record. The sequence decoder analyzes the temporal correlation between multiple comprehensive feature vectors to obtain the context memory vector of the target transaction sequence at the target time step based on the temporal correlation. The context memory vector is used to perform a classification task on the target transaction sequence. Because all transaction features of each transaction record in the transaction sequence are vectorized, the independent attributes and relationships of each feature within a single transaction are fully preserved, achieving effective fusion of structured data and natural language text. Therefore, this application can process the feature tensor of the transaction sequence through a feature encoder to encode the relationship between feature vectors within a single transaction to mine the semantic logic within the transaction. Then, the sequence decoder analyzes the temporal relationship between the comprehensive feature vectors of multiple transactions and generates a context memory vector. With the hierarchical division of labor and cooperation between the feature encoder and the sequence decoder, the hierarchical processing significantly improves the fineness and richness of the transaction representation, which can meet the needs of deep understanding of the transaction sequence. At the same time, applying the context memory vector to the transaction sequence classification task effectively improves the understanding of transaction behavior and the accuracy of classification.

[0026] In some embodiments, in S110, the target transaction sequence is a set of multiple transaction records, which can be all transaction records belonging to the same user or the same account within a sliding time window. Each transaction record contains multiple fields describing the transaction content, each field being a transaction feature, such as transaction amount, merchant type, transaction occurrence time, merchant name, etc. Each transaction record is composed of multiple homogeneous or heterogeneous transaction features. The end time of the sliding event window is no later than the current time, and the start time can be determined based on the duration and end time of the sliding time window. The times can be set as needed, and the start and end times can change dynamically; this embodiment does not limit this.

[0027] For example, if the sliding time window lasts for one year and ends on January 1, 2026, then the starting time is January 1, 2025.

[0028] For each transaction record in the target transaction sequence, each transaction feature in the transaction record is vectorized to obtain a feature vector sequence for each transaction record. The feature vector sequence of each transaction record includes the feature vectors corresponding to each of the transaction features.

[0029] When vectorizing each transaction feature in a transaction record, the key-value pairs of each transaction feature can be tokenized and encoded first, and then processed through vector embedding.

[0030] In some embodiments, the specific processing steps of S110 may include: For each key-value pair of a transaction feature in the transaction record, the key and value in the key-value pair are tokenized and encoded to obtain the feature attribute tag and feature value tag corresponding to the key-value pair; For each key-value pair of a transaction feature, a structured tag combination of the transaction feature is generated based on the feature attribute tag, the feature value tag, and the feature association tag corresponding to the key-value pair. The structured tag combination is then vector-embedded to generate the feature vector corresponding to the transaction feature. Based on the feature vectors corresponding to each transaction feature, a sequence of feature vectors for each transaction record is constructed.

[0031] In some embodiments, a key-value pair of a transaction feature is a structured data unit that represents a certain attribute in a transaction record, wherein the key represents the feature attribute of the transaction feature and the value represents the feature value of the transaction feature. For example, in “Transaction Amount: 100 yuan”, “Transaction Amount” is the key and “100 yuan” is the value.

[0032] For each key-value pair of transaction features in a single transaction record, tokenization encoding needs to be performed on the two independent units of key and value to convert them into discrete tokens, thereby obtaining feature attribute tokens and feature value tokens. Among them, feature attributes (such as "merchant type") are tokenized and encoded as independent semantic units, just like feature values.

[0033] The following encoding method can be used: For categorical keys / values ​​(such as "merchant type" or "catering category"), the corresponding discrete token is found through the embedding layer to obtain the feature attribute token / feature value token; For numeric keys / values ​​(such as transaction amount "100 yuan"), first discretize them into preset categories using binning, and then match the corresponding discrete tokens (Tokens) to obtain feature attribute tokens / feature value tokens; For text-based key / value pairs (such as the merchant name "XX Convenience Store"), the Trie tree segmenter is used to divide them into sub-word sequences, and then the corresponding token sequences are matched to obtain feature attribute tokens and feature value tokens.

[0034] Each key-value pair of a transaction feature corresponds to a unique set of feature attribute labels and feature value labels, thus completing the symbolic representation of a single feature.

[0035] In some embodiments, feature association markers are markers used to construct the key-value pair semantic structure of transaction features. Each feature association marker corresponds to a specific identifier character, such as the category start marker corresponding to the start category character [CLS], the equivalence relation marker corresponding to the equivalence relation character [EQ], the feature separator marker corresponding to the feature separator character [SEP], etc.

[0036] For each key-value pair of a transaction feature in a single transaction record, a feature association tag is introduced based on the feature attribute tag and feature value tag corresponding to the key-value pair. The feature association tag, feature attribute tag, and feature value tag are concatenated according to preset syntax rules, for example, "tag corresponding to [CLS] + feature attribute tag + tag corresponding to [EQ] + feature value tag + tag corresponding to [SEP]", forming a structured tag combination for a single transaction feature, thereby clarifying the semantic correspondence between "feature attribute" and "feature value".

[0037] For a single transaction feature, the generated structured labels are combined and input into a pre-trained embedding layer. The discrete label sequence is mapped into a fixed-dimensional numerical vector through vector embedding operations. This numerical vector is the feature vector corresponding to a single transaction feature. The vector integrates the feature's attribute semantics, value information, and structural associations.

[0038] For a single transaction record, feature vectors corresponding to all transaction characteristics of that single transaction record are collected. All feature vectors are then arranged, and the resulting set of arranged vectors is the feature vector sequence of that transaction record. The arrangement can be random or in a preset order; this embodiment does not impose any restrictions on this.

[0039] When vectorizing each transaction, each transaction feature is split into two independent units: feature attribute and feature value. Each feature attribute and feature value is treated as an independent semantic unit, corresponding to a separate tag, achieving fine-grained representation where "a single transaction feature corresponds to a single feature vector." This not only preserves the semantic information of the original transaction to the maximum extent, avoiding the feature association loss problem caused by the atomic processing of the entire transaction in existing technologies, but also significantly reduces the size of the transaction dictionary. Simultaneously, it achieves effective fusion of structured data and natural language text. Therefore, each transaction record is mapped to a token sequence, and the entire target transaction sequence is mapped to a token matrix. Where x is the Token matrix, L is the transaction sequence length, F is the total number of all feature attributes and all feature values, and V is the vocabulary set. The Token matrix is ​​shown in Figure 2. The individual features shown in the figure are feature attributes or feature values ​​in the transaction features.

[0040] In some embodiments, in S120, the target feature tensor is high-dimensional structured data formed by fusing the feature vector sequence and time series information of all transaction records in the target transaction sequence.

[0041] The process of generating the target feature tensor can be as follows: First, sort the feature vector sequences of all transaction records in ascending order according to the transaction occurrence time of each transaction record to ensure the correct temporal order; then, stack the sorted feature vector sequences according to the "transaction quantity" dimension to form a three-dimensional matrix; finally, through tensor dimension transformation, standardize the three-dimensional matrix into a target feature tensor of "transaction quantity × number of features per transaction × feature vector dimension". , where E is the target feature tensor, R is the real number field, and D is the number of vector embedding dimensions.

[0042] In some embodiments, in S130, the feature encoder is a model module built on a self-attention mechanism, used to mine the correlation between feature vectors within a single transaction record.

[0043] The feature encoder can adopt a multi-layer Transformer encoder architecture, or other architectures, such as a convolutional neural network + self-attention hybrid architecture, etc. This embodiment does not limit this.

[0044] The target feature tensor is input into the feature encoder, which encodes the correlation between the feature vectors in the feature vector sequence of each transaction record, thus obtaining a comprehensive feature vector representing each transaction record that contains rich internal structural information.

[0045] In some embodiments, in S140, the sequence decoder is a model module built on a temporal self-attention mechanism, used to mine the temporal correlation between multiple transaction records.

[0046] The sequence decoder can adopt a multi-layer Transformer decoder architecture, or other architectures, such as a hybrid architecture of Long Short-Term Memory (LSTM) network + self-attention mechanism, etc. This embodiment does not limit this.

[0047] The comprehensive feature vector of each transaction record is input into the sequence decoder in sequence according to the transaction occurrence time of each transaction record. The sequence decoder analyzes the temporal correlation between multiple comprehensive feature vectors and learns the evolution pattern of the behavior of the object to which the target transaction sequence belongs.

[0048] When encoding transaction records at time step t, the sequence decoder focuses only on all fused information up to the target time step t, thereby mining temporal correlations and obtaining the context memory vector of the target transaction sequence at the target time step t based on the temporal correlations. The context memory vector represents a condensed summary of all historical information up to the target time step t.

[0049] The context memory vector of the target transaction sequence at the target time step can be directly used for subsequent downstream tasks such as classification and risk assessment of the target transaction sequence.

[0050] In some embodiments, when performing S130, Figure 3 A schematic diagram of an encoding process provided for an embodiment of this application, such as Figure 3 As shown.

[0051] S310: Input the target feature tensor into the feature encoder. The input tensor is encoded layer by layer through multiple self-attention layers stacked in the feature encoder to obtain the output feature tensor of the last self-attention layer in the feature encoder.

[0052] S320: Extract the feature representation of each transaction record from the output feature tensor of the last self-attention layer to obtain the comprehensive feature vector of each transaction record.

[0053] In some embodiments, in S310, the feature encoder employs a multi-layer Transformer encoder architecture, comprising multiple self-attention layers arranged in a stacked manner.

[0054] The self-attention layer contains computational logic such as self-attention computation, residual connections, and normalization, which is used to capture the correlation information of elements in the input tensor.

[0055] The target feature tensor is used as the input tensor of the first self-attention layer in the feature encoder. The target feature tensor has fused the temporal information of the transaction sequence with the feature vector sequence of a single transaction record.

[0056] Specifically, each self-attention layer in the feature encoder calculates the correlation between all tokens (including feature attribute tokens, feature value tokens, and feature association tokens for each transaction feature) in the input tensor through a bidirectional self-attention mechanism. Simultaneously, after each layer's calculation, the output is optimized through residual connections and normalization operations to obtain the output feature tensor of that layer. The output feature tensor of the previous self-attention layer is used as the input tensor of the next self-attention layer, and the encoding of all self-attention layers is completed sequentially.

[0057] The output feature tensor of each self-attention layer can be specifically expressed as follows: ; ; ; in, Let be the output feature tensor of the i-th self-attention layer of the feature encoder. ; m is the total number of layers in the feature encoder; LayerNorm is the normalization operation; Concat is the concatenation operation; Softmax is the activation function; The output weight matrix for multi-head attention; , , These are the weight matrices for the query, key, and value, respectively. The dimension of the query / key vector.

[0058] In some embodiments, after completing multi-layer self-attention encoding, a comprehensive feature vector corresponding to each transaction record is extracted from the output feature tensor of the last self-attention layer of the feature encoder. Since the feature representation of each tag is integrated into the overall context information of the transaction record after encoding, the feature representation of any tag in the corresponding feature vector sequence of the transaction record can be selected as the comprehensive feature vector. In this embodiment, the feature representation corresponding to the tag at the beginning of the feature vector sequence (e.g., the category start tag) can be selected.

[0059] In this embodiment, a multi-layer self-attention architecture feature encoder is used to encode the target feature tensor that integrates temporal information layer by layer, mining the correlation between features within a single transaction. Then, based on the encoded output feature tensor, a global feature representation is extracted as a comprehensive feature vector. This not only achieves fine-grained structured representation of transaction features, but also allows the comprehensive feature vector to integrate the complete contextual semantics within the transaction, providing a high-density, high-information feature foundation for subsequent mining of temporal correlation between transaction sequences.

[0060] In some embodiments, when performing S140, Figure 4 A schematic diagram of an analysis process provided for an embodiment of this application, such as Figure 4 As shown.

[0061] S410: Input the comprehensive feature vector of each transaction record into the sequence decoder in sequence according to the transaction occurrence time of each transaction record. The input tensor is encoded layer by layer by multiple self-attention layers stacked in the sequence decoder to obtain the output feature tensor of the last self-attention layer in the sequence decoder.

[0062] S420: Extract the feature representation of the target transaction record from the output feature tensor of the last self-attention layer to obtain the context memory vector of the target transaction sequence at the target time step.

[0063] In some embodiments, in S410, the sequence decoder employs a multi-layer Transformer decoder architecture, comprising multiple self-attention layers arranged in a stacked manner.

[0064] The self-attention layer contains computational logic such as self-attention calculation, residual connection, and normalization, which is used to capture the temporal correlation between the comprehensive feature vectors of each transaction record.

[0065] The composite feature vector of each transaction record is arranged sequentially according to the transaction occurrence time and input into the sequence decoder. The input tensor of the first self-attention layer in the sequence decoder is composed of the composite feature vectors of all historical transaction records up to the target time step t, arranged according to the transaction occurrence time, thus ensuring temporal causality.

[0066] Specifically, each self-attention layer in the sequence decoder is computed using an autoregressive masked attention mechanism: when encoding transaction records at the target time step, it only focuses on the comprehensive feature vectors of the target time step and earlier, capturing the temporal correlation between the comprehensive feature vectors of each transaction record; simultaneously, after each layer's computation, the output result is optimized through residual connections and normalization operations to obtain the output feature tensor of that layer. The output feature tensor of the previous self-attention layer is used as the input tensor of the next self-attention layer, and the encoding of all self-attention layers is completed sequentially.

[0067] After completing the multi-layer self-attention encoding, the vector corresponding to the tensor position of the target time step is extracted from the output feature tensor of the last self-attention layer of the sequence decoder, thus obtaining the context memory vector of the target transaction sequence at the target time step. The vector at this position has fused the information interaction results of the target time step and all previous time steps. Therefore, the context memory vector is a dynamic aggregate of all historical sequence information.

[0068] In this embodiment, by encoding the integrated feature vector layer by layer into the stacked self-attention layers in strict time sequence, the model can fully learn the complex dependencies across time in the transaction sequence; finally, the encoding result of the target time step is extracted to generate a context memory vector that condenses all historical transaction information up to that moment, providing an accurate global historical representation for subsequent analysis.

[0069] In some embodiments, the sequence decoder encodes the input tensor through a self-attention layer, specifically including: By using multiple attention heads in the self-attention layer, causal mask attention operations are performed on the input tensor respectively to obtain the operation results of each attention head. The operation results of each attention head are then concatenated and mapped to obtain the multi-head self-attention output. The multi-head self-attention output is residually concatenated with the input tensor to obtain the concatenation result; Perform layer normalization on the connection result to obtain the output feature tensor of the self-attention layer.

[0070] The self-attention layer in the sequence decoder is designed with a multi-head attention mechanism, which includes computational logic such as causal mask attention operation, residual connection, and layer normalization.

[0071] Of course, each self-attention layer of the sequence decoder may also include other computational logic, and this embodiment does not limit this.

[0072] An attention head is an independent attention computation submodule within a self-attention layer. Parallel computation of multiple attention heads can mine the correlation features of the input tensor from different dimensions, improving the model's ability to model complex information. Causal masking attention computation is a self-attention computation method with temporal constraints. It uses a mask to block information after the target time step, ensuring that the computation only focuses on the input of the target time step and the historical time steps before it.

[0073] In some embodiments, when each self-attention layer of the sequence decoder encodes the input tensor, the specific process is as follows: the input tensor of the self-attention layer is simultaneously assigned to multiple independent attention heads, and each attention head performs computation in parallel; each attention head independently performs causal mask attention operation on the input tensor to obtain the computation result of a single attention head.

[0074] After obtaining the computation results of all attention heads, the data is concatenated according to the feature dimension. Then, through a linear mapping layer, the concatenated high-dimensional feature data is converted into feature data that matches the dimension of the input tensor, resulting in multi-head self-attention output. The multi-head self-attention output is superimposed with the input tensor of the current self-attention layer to achieve residual connection, resulting in connection result. The connection result retains both the temporal correlations mined by the self-attention layer and the basic information of the original input tensor.

[0075] A layer normalization operation is performed on the connection result to obtain the output feature tensor of the current self-attention layer. Through the synergistic effect of the above-mentioned causal mask multi-head attention operation, residual connection, and layer normalization operation, a stable and efficient modeling of complex temporal relationships in the input tensor is achieved, laying the foundation for generating high-quality context memory vectors.

[0076] The output feature tensor of each self-attention layer can be specifically expressed as follows: ; in, Let be the output feature tensor of the i-th self-attention layer of the sequence decoder. n is the total number of layers in the sequence decoder.

[0077] In some embodiments, Figure 5 This embodiment provides an overall flowchart for time-series data processing of a target transaction sequence using a feature encoder-sequence decoder. The following is a description of... Figure 5 The specific details will be explained in detail.

[0078] The transaction features (including feature attributes, feature values, and feature association markers) corresponding to transaction records 1 to t within the visible window are input into the feature encoder. Through multi-layer self-attention calculation of the feature encoder, the comprehensive feature vector corresponding to each transaction record is obtained (i.e., the feature representation corresponding to the category start marker [CLS] in the structured marker combination). At the same time, the Masked Language Model (MLM) task is performed and the MLM loss is calculated during this process. Among them, e1-Token to et-Token are the token sequence identifiers corresponding to each transaction record 1 to transaction record t within the visible window, which are used to associate the comprehensive feature vector of each transaction with the corresponding time step. Subsequently, the comprehensive feature vectors of each transaction record are input into a sequence transformer with causal masking (i.e., a sequence decoder). The sequence decoder encodes the temporal information through an autoregressive self-attention mechanism. Finally, the output of the sequence decoder combines the information of transaction records t+1 to t+k within the future window, calculates the dense multi-step prediction loss based on the comprehensive feature vectors of each transaction associated with e1-Token to et-Token, and outputs the context memory vector corresponding to the target time step t, thereby completing the temporal data processing of the target transaction sequence.

[0079] In some embodiments, the first loss function corresponding to the feature encoder determines the loss value based on the output feature label of the mask feature in the sample feature tensor and the original label of the mask feature in the sample feature tensor; The second loss function corresponding to the sequence decoder determines the loss value based on the predicted feature vector of the predicted transaction record and the sample feature vector of the sample transaction record corresponding to the predicted transaction record. The predicted feature vector of the predicted transaction record is obtained by the sequence decoder based on the temporal correlation between the sample output vectors of each sample transaction record in the sample transaction sequence.

[0080] Masked features refer to features that are randomly selected and masked in the sample feature tensor. They can be feature attributes, feature values, or feature association labels of a single transaction feature in a sample transaction record, and are used for training the Masked Language Model (MLM) task. Output labels refer to the labeling results output by the feature encoder after predicting the masked features. Original labels are the original labels (Tokens) of the masked features when they are not masked in the sample feature tensor.

[0081] The first loss function corresponding to the feature encoder is as follows: ; in, The vector (i.e., output label) of the f-th feature of the l-th sample transaction record output by the feature encoder. This is the original label corresponding to the f-th mask feature of the l-th sample transaction record. The number of mask features.

[0082] The first loss function, by calculating the difference between the predicted probability of the mask features by the feature encoder and the original label, can accurately measure the modeling accuracy of the feature encoder in terms of the correlation between internal features of a transaction.

[0083] A predicted transaction sequence refers to a transaction record at a future time step predicted by the sequence decoder based on historical information up to the time step corresponding to the l-th sample transaction record in the sample transaction sequence and before. The predicted feature vector is the feature vector output by the sequence decoder that corresponds to the predicted transaction record. It is generated by the sequence decoder after mining the temporal correlation of the sample transaction sequence and contains a prediction token of a single feature.

[0084] The second loss function corresponding to the sequence decoder is as follows: ; in, This is the vector of the l-th sample transaction record output by the feature encoder. Let L be the original token corresponding to each feature of the (l+1)th sample transaction record in the sample transaction sequence (i.e., the real sample corresponding to the predicted transaction record), and L be the length of the sample transaction sequence.

[0085] The second loss function is calculated by the sequence decoder based on historical transaction vectors. The difference between the predicted probability of future transaction tags and the actual tags can accurately measure the modeling accuracy of the sequence decoder in terms of the temporal correlation of transaction sequences.

[0086] It should be noted that the feature encoder and sequence decoder together constitute a hierarchical self-attention model, which is jointly trained through a joint loss function: the feature encoder is self-supervised by a masked language model, which endows the model with the ability to model the correlation between transactions and features; the sequence decoder is self-supervised by Next Step Prediction, which endows the model with the ability to model the correlation between sequences and transactions.

[0087] The joint loss function is as follows: ; Among them, weight These are model hyperparameters, which can be set as needed, and the embodiments in this application do not impose any restrictions on them.

[0088] The joint loss function optimizes the overall modeling capability of the model by weighted summation of the feature encoder loss and the sequence decoder loss.

[0089] Figure 6 This application provides a schematic diagram of a model training process, as shown in the embodiment of the present application. Figure 6 As shown.

[0090] The feature encoder and sequence decoder are trained based on the following steps: S610: Obtain the sample transaction sequences of multiple sample objects and the sample feature tensors of each sample transaction sequence.

[0091] S620: For each sample transaction sequence, perform random masking on the sample feature tensor of the sample transaction sequence to obtain the masked features and the masked sample feature tensor.

[0092] S630: For each sample transaction sequence, input the masked sample feature tensor into the feature encoder to be trained to obtain the sample output vector of each sample transaction record in the sample transaction sequence, and determine the first loss value corresponding to the first loss function based on the output feature label of the masked feature in the sample output vector and the original label of the masked feature in the sample feature tensor.

[0093] S640: Input the context representation vector of each sample transaction record in the sample transaction sequence into the sequence decoder to be trained in sequence according to the transaction occurrence time of each sample transaction record. Analyze the temporal correlation between multiple context representation vectors through the sequence decoder to be trained, so as to obtain the prediction feature vector of the predicted transaction record based on the temporal correlation.

[0094] S650: For each sample transaction sequence, determine the second loss value corresponding to the second loss function based on the predicted feature vector of the predicted transaction record and the sample feature vector of the sample transaction record corresponding to the predicted transaction record.

[0095] S660: Determine the target loss value corresponding to the joint loss function based on the first loss value corresponding to the first loss function and the second loss value corresponding to the second loss function.

[0096] S670: Based on the target loss value corresponding to the joint loss function, iteratively optimize the parameters of the feature encoder and the sequence decoder to be trained until the target loss value meets the preset convergence condition.

[0097] In some embodiments, in S610, a sample object refers to a subject (e.g., a user or account) that provides historical transaction data, and each sample object corresponds to a sample transaction sequence; the sample feature tensor is generated based on the sample feature vector of each sample transaction record in the sample transaction sequence, combined with its transaction occurrence time.

[0098] To ensure the accuracy of subsequent model training, padding is performed on all sample feature tensors to ensure that the sequence lengths corresponding to each sample feature tensor are consistent.

[0099] For each sample transaction sequence, some features corresponding to tokens are randomly selected from its corresponding sample feature tensor as mask features, which are then masked to obtain the masked sample feature tensor, which is used for prediction training of the subsequent feature encoder.

[0100] The masking method can be to replace the feature with a preset mask symbol. Of course, other masking methods can also be used, and this embodiment does not limit this.

[0101] In some embodiments, the masked sample feature tensor is input into the feature encoder to be trained; the feature encoder to be trained calculates the correlation between all tokens (including feature attribute tokens, feature value tokens, and feature association tokens of each transaction feature) in the tensor through a bidirectional self-attention mechanism, and outputs the sample output vector corresponding to each sample transaction record.

[0102] Next, the output feature labels corresponding to the mask features in the sample output vector are extracted and compared with the original labels of the mask features in the sample feature tensor. The difference between the two is calculated using the first loss function to obtain the first loss value.

[0103] The context representation vector of each sample transaction record in the sample transaction sequence, output by the feature encoder to be trained, is sequentially input into the sequence decoder to be trained in chronological order of transaction occurrence. The sequence decoder uses an autoregressive self-attention mechanism to focus only on the context representation vectors at and before time step t when encoding the sample transaction record l at time step t, thereby mining the temporal correlation between transactions. Based on the temporal correlation, the predicted feature vector corresponding to the predicted transaction record at future time steps is generated.

[0104] Among them, the context representation vector of the sample transaction record is a comprehensive feature vector that represents the sample transaction record and contains rich internal structural information, and the context representation vector is a feature representation located at a preset position in the sample output vector of the sample transaction record.

[0105] Optionally, the preset position can be the first position in the sample output vector. For example, the vector corresponding to the marker at the beginning of the sequence in the sample output vector (e.g., the category start marker) can be used as the context representation vector. Other positions can also be selected as the preset position. This embodiment does not limit this.

[0106] In some embodiments, in S650, the sample transaction record corresponding to the predicted transaction record refers to the sample transaction record that actually exists in the sample transaction sequence at the next time step t+1.

[0107] From the sample feature tensor of the sample transaction sequence, obtain the sample feature vector of the sample transaction record corresponding to the predicted transaction record.

[0108] Then, the predicted feature vector is normalized to obtain the predicted probability distribution of future transaction tags; the difference between the predicted probability distribution and each original tag in the sample feature vector of the sample transaction record corresponding to the predicted transaction record is calculated by the second loss function to obtain the second loss value.

[0109] The first and second loss values ​​are weighted and summed to obtain the target loss value corresponding to the joint loss function, which is used for iterative optimization of the model parameters.

[0110] Specifically, the preset convergence condition refers to the pre-defined stopping condition during model training, including but not limited to any of the following: the total loss value is lower than a threshold, or the number of iterations reaches the upper limit. Training stops when any of the preset convergence conditions is met.

[0111] Based on the target loss value, the parameters of the feature encoder and sequence decoder are iteratively updated using optimization algorithms such as gradient descent. Steps S620 to S660 are repeated until the target loss value meets the preset convergence condition, or training stops when the number of iterations reaches the upper limit, thus obtaining the trained feature encoder and sequence decoder.

[0112] In this embodiment, a hierarchical self-supervised training framework is used to enable the feature encoder to accurately learn the semantic and logical relationships between features within a single transaction through a mask language modeling task, while the sequence decoder effectively captures the dynamic temporal dependencies between transaction sequences through a next event prediction task. The two are jointly optimized through a joint loss function, ultimately resulting in an end-to-end hierarchical self-attention model that takes into account both single transaction feature extraction and sequence temporal evolution modeling.

[0113] Based on the time-series data processing method provided in the above embodiments, this application also provides specific implementations of a time-series data processing apparatus. Please refer to the following embodiments.

[0114] like Figure 7As shown, the timing data processing apparatus 700 provided in this application embodiment includes the following modules: The transaction feature processing module 701 is used to vectorize each transaction feature in each transaction record in the target transaction sequence to obtain a feature vector sequence for each transaction record, wherein the feature vector sequence includes the feature vectors corresponding to each transaction feature. The feature tensor generation module 702 is used to generate a target feature tensor of the target transaction sequence based on the feature vector sequence of each transaction record and the transaction occurrence time of each transaction record. The feature vector encoding module 703 is used to input the target feature tensor into the feature encoder, and to encode the correlation between each feature vector in the feature vector sequence of each transaction record to obtain the comprehensive feature vector of each transaction record. The memory vector processing module 704 is used to input the comprehensive feature vector of each transaction record into the sequence decoder in sequence according to the transaction occurrence time of each transaction record. The sequence decoder analyzes the temporal correlation between multiple comprehensive feature vectors to obtain the context memory vector of the target transaction sequence at the target time step based on the temporal correlation. The context memory vector is used to perform a classification task on the target transaction sequence.

[0115] The timing data processing device 700 described above will be described in detail below: Optionally, the transaction feature processing module 701 is specifically used to perform tokenized encoding on the key and value of each key-value pair of the transaction feature in the transaction record to obtain the feature attribute tag and feature value tag corresponding to the key-value pair. In the key-value pair of the transaction feature, the key represents the feature attribute of the transaction feature, and the value represents the feature value of the transaction feature. For each key-value pair of the transaction feature, a structured tag combination of the transaction feature is generated based on the feature attribute tag corresponding to the key-value pair, the feature value tag corresponding to the key-value pair, and the feature association tag. Vector embedding is then performed on the structured tag combination to generate the feature vector corresponding to the transaction feature. Based on the feature vectors corresponding to each of the aforementioned transaction features, a sequence of feature vectors for the transaction records is constructed.

[0116] Optionally, the feature vector encoding module 703 is specifically used to input the target feature tensor into the feature encoder, and encode the input tensor layer by layer through multiple self-attention layers stacked in the feature encoder to obtain the output feature tensor of the last self-attention layer in the feature encoder. The input tensor of the first self-attention layer in the feature encoder is the target feature tensor, and the input tensor of the next self-attention layer in the feature encoder is the output feature tensor of the previous self-attention layer. From the output feature tensor of the last self-attention layer, the feature representation of each transaction record is extracted to obtain the comprehensive feature vector of each transaction record.

[0117] Optionally, the memory vector processing module 704 is specifically used to input the comprehensive feature vector of each transaction record into the sequence decoder in sequence according to the transaction occurrence time of each transaction record, and encode the input tensor layer by layer through multiple self-attention layers stacked in the sequence decoder to obtain the output feature tensor of the last self-attention layer in the sequence decoder. From the output feature tensor of the last self-attention layer, the feature representation of the target transaction record is extracted to obtain the context memory vector of the target transaction sequence at the target time step; In the sequence decoder, the input tensor of the first self-attention layer is composed of the comprehensive feature vector of each transaction record, which is input sequentially according to the transaction occurrence time of each transaction record up to the target time step; the input tensor of the next self-attention layer in the sequence decoder is the output feature tensor of the previous self-attention layer.

[0118] Optionally, the memory vector processing module 704 is specifically used to perform causal mask attention operations on the input tensor through multiple attention heads in the self-attention layer, obtain the operation results of each attention head, and concatenate and map the operation results of each attention head to obtain a multi-head self-attention output; The multi-head self-attention output is residually concatenated with the input tensor to obtain the concatenation result; Perform layer normalization on the connection result to obtain the output feature tensor of the self-attention layer.

[0119] Optionally, the device further includes: Obtain the sample transaction sequences of multiple sample objects and the sample feature tensors of each sample transaction sequence; The masking module is used to perform random masking on the sample feature tensor of each sample transaction sequence to obtain masked features and masked sample feature tensors. The first loss value calculation module is used to input the masked sample feature tensor into the feature encoder to be trained for each sample transaction sequence to obtain the sample output vector of each sample transaction record in the sample transaction sequence, and determine the first loss value corresponding to the first loss function based on the output feature label of the masked feature in the sample output vector and the original label of the masked feature in the sample feature tensor. The feature vector prediction module is used to, for each sample transaction sequence, input the context representation vector of each sample transaction record in the sample transaction sequence into the sequence decoder to be trained in sequence according to the transaction occurrence time of each sample transaction record. The sequence decoder to be trained analyzes the temporal correlation between multiple context representation vectors to obtain the predicted feature vector of the predicted transaction record based on the temporal correlation. The context representation vector of the sample transaction record is the feature representation located at a preset position in the sample output vector of the sample transaction record. The second loss value calculation module is used to determine the second loss value corresponding to the second loss function for each sample transaction sequence, based on the predicted feature vector of the predicted transaction record and the sample feature vector of the sample transaction record corresponding to the predicted transaction record. The target loss value calculation module is used to determine the target loss value corresponding to the joint loss function based on the first loss value corresponding to the first loss function and the second loss value corresponding to the second loss function; The iterative optimization module is used to iteratively optimize the parameters of the feature encoder and the sequence decoder to be trained based on the target loss value corresponding to the joint loss function, until the target loss value meets the preset convergence condition.

[0120] In this embodiment, for each transaction record in the target transaction sequence, each transaction feature in the transaction record is vectorized to obtain a feature vector sequence for each transaction record. The feature vector sequence includes a sequence of feature vectors corresponding to each transaction feature. Based on the feature vector sequence of each transaction record and the transaction occurrence time of each transaction record, a target feature tensor of the target transaction sequence is generated. The target feature tensor is input into a feature encoder, which encodes the correlation between the feature vectors in the feature vector sequence of each transaction record to obtain a comprehensive feature vector for each transaction record. The comprehensive feature vector of each transaction record is input into a sequence decoder according to the transaction occurrence time of each transaction record. The sequence decoder analyzes the temporal correlation between multiple comprehensive feature vectors to obtain the context memory vector of the target transaction sequence at the target time step based on the temporal correlation. The context memory vector is used to perform a classification task on the target transaction sequence. Because all transaction features of each transaction record in the transaction sequence are vectorized, the independent attributes and relationships of each feature within a single transaction are fully preserved, achieving effective fusion of structured data and natural language text. Therefore, this application can process the feature tensor of the transaction sequence through a feature encoder to encode the relationship between feature vectors within a single transaction to mine the semantic logic within the transaction. Then, the sequence decoder analyzes the temporal relationship between the comprehensive feature vectors of multiple transactions and generates a context memory vector. With the hierarchical division of labor and cooperation between the feature encoder and the sequence decoder, the hierarchical processing significantly improves the fineness and richness of the transaction representation. At the same time, the context memory vector is applied to the transaction sequence classification task, effectively improving the understanding of transaction behavior and the accuracy of classification.

[0121] Based on the timing data processing method provided in the above embodiments, this application also provides specific implementation methods for electronic devices. Figure 8 A schematic diagram of an electronic device 800 provided in an embodiment of this application is shown.

[0122] Electronic device 800 may include processor 810 and memory 820 storing computer program instructions.

[0123] Specifically, the processor 810 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0124] Memory 820 may include mass storage for data or instructions. For example, and not limitingly, memory 820 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 820 may include removable or non-removable (or fixed) media. Where appropriate, memory 820 may be internal or external to electronic device 800. In a particular embodiment, memory 820 is a non-volatile solid-state memory.

[0125] In a specific embodiment, the memory 820 can be implemented as ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 820 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 820 and called and executed by the processor 810. The processor 810 implements any of the timing data processing methods in the above embodiments by reading and executing the computer program instructions stored in the memory 820.

[0126] The processor 810 reads and executes computer program instructions stored in the memory 820 to implement any of the timing data processing methods in the above embodiments.

[0127] In one example, the electronic device 800 may also include a communication interface 830 and a bus 840. For example, Figure 8 As shown, the processor 810, memory 820, and communication interface 830 are connected through bus 840 and complete communication with each other.

[0128] The communication interface 830 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0129] Bus 840 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 840 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0130] For example, the electronic device 800 can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc.

[0131] The electronic device can execute the timing data processing method in the embodiments of this application, thereby achieving the combination Figures 1 to 7 The time-series data processing method described herein, and the beneficial effects of the corresponding method embodiments, will not be elaborated further here.

[0132] Furthermore, in conjunction with the timing data processing methods in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the timing data processing methods in the above embodiments.

[0133] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0134] The computer program instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the timing data processing method as shown in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0135] Based on the timing data processing methods in the above embodiments, this application can provide a computer program product for implementation. When the instructions in this computer program product are executed by the processor of an electronic device, they implement any of the timing data processing methods in the above embodiments.

[0136] The computer program products of the above embodiments are used to implement the timing data processing method shown in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0137] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0138] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0139] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0140] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0141] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A time-series data processing method, characterized in that, include: For each transaction record in the target transaction sequence, each transaction feature in the transaction record is vectorized to obtain a feature vector sequence for each transaction record, wherein the feature vector sequence includes the feature vectors corresponding to each transaction feature. Based on the feature vector sequence of each transaction record and the transaction occurrence time of each transaction record, generate the target feature tensor of the target transaction sequence; The target feature tensor is input into the feature encoder, and the feature encoder encodes the correlation between each feature vector in the feature vector sequence of each transaction record to obtain the comprehensive feature vector of each transaction record. The comprehensive feature vector of each transaction record is sequentially input into the sequence decoder according to the transaction occurrence time of each transaction record. The sequence decoder analyzes the temporal correlation between multiple comprehensive feature vectors to obtain the context memory vector of the target transaction sequence at the target time step based on the temporal correlation. The context memory vector is used to perform a classification task on the target transaction sequence.

2. The method according to claim 1, characterized in that, Each transaction feature in the transaction record is vectorized to obtain a feature vector sequence for each transaction record, including: For each key-value pair of the transaction feature in the transaction record, the key and value in the key-value pair are respectively tokenized and encoded to obtain the feature attribute tag and feature value tag corresponding to the key-value pair. In the key-value pair of the transaction feature, the key represents the feature attribute of the transaction feature, and the value represents the feature value of the transaction feature. For each key-value pair of the transaction feature, a structured tag combination of the transaction feature is generated based on the feature attribute tag corresponding to the key-value pair, the feature value tag corresponding to the key-value pair, and the feature association tag. Vector embedding is then performed on the structured tag combination to generate the feature vector corresponding to the transaction feature. Based on the feature vectors corresponding to each of the aforementioned transaction features, a sequence of feature vectors for the transaction records is constructed.

3. The method according to claim 1, characterized in that, The target feature tensor is input into a feature encoder, which encodes the correlation between feature vectors within the feature vector sequence of each transaction record to obtain a comprehensive feature vector for each transaction record, including: The target feature tensor is input into the feature encoder, and the input tensor is encoded layer by layer through multiple stacked self-attention layers in the feature encoder to obtain the output feature tensor of the last self-attention layer in the feature encoder. The input tensor of the first self-attention layer in the feature encoder is the target feature tensor, and the input tensor of the next self-attention layer in the feature encoder is the output feature tensor of the previous self-attention layer. From the output feature tensor of the last self-attention layer, the feature representation of each transaction record is extracted to obtain the comprehensive feature vector of each transaction record.

4. The method according to claim 1, characterized in that, The comprehensive feature vector of each transaction record is sequentially input into the sequence decoder according to the transaction occurrence time of each transaction record. The sequence decoder analyzes the correlation between multiple comprehensive feature vectors to obtain the context memory vector of the target transaction sequence at the target time step based on the correlation, including: The comprehensive feature vector of each transaction record is input into the sequence decoder in sequence according to the transaction occurrence time of each transaction record. The input tensor is encoded layer by layer through multiple self-attention layers stacked in the sequence decoder to obtain the output feature tensor of the last self-attention layer in the sequence decoder. From the output feature tensor of the last self-attention layer, the feature representation of the target transaction record is extracted to obtain the context memory vector of the target transaction sequence at the target time step; In the sequence decoder, the input tensor of the first self-attention layer is composed of the comprehensive feature vector of each transaction record, which is input sequentially according to the transaction occurrence time of each transaction record up to the target time step; the input tensor of the next self-attention layer in the sequence decoder is the output feature tensor of the previous self-attention layer.

5. The method according to claim 4, characterized in that, The input tensor is encoded using a self-attention layer, including: The self-attention layer uses multiple attention heads to perform causal masking attention operations on the input tensor, obtaining the operation results of each attention head. The operation results of each attention head are then concatenated and mapped to obtain the multi-head self-attention output. The multi-head self-attention output is residually concatenated with the input tensor to obtain the concatenation result; Perform layer normalization on the connection result to obtain the output feature tensor of the self-attention layer.

6. The method according to any one of claims 1 to 5, characterized in that, The first loss function corresponding to the feature encoder determines the loss value based on the output feature label of the mask feature in the sample feature tensor and the original label of the mask feature in the sample feature tensor; The second loss function corresponding to the sequence decoder determines the loss value based on the predicted feature vector of the predicted transaction record and the sample feature vector of the sample transaction record corresponding to the predicted transaction record. The predicted feature vector of the predicted transaction record is obtained by the sequence decoder based on the temporal correlation between the sample output vectors of each sample transaction record in the sample transaction sequence.

7. The method according to claim 6, characterized in that, The training process of the feature encoder and the sequence decoder includes: Obtain the sample transaction sequences of multiple sample objects and the sample feature tensors of each sample transaction sequence; For each of the sample transaction sequences, a random masking process is performed on the sample feature tensor of the sample transaction sequence to obtain the masked features and the masked sample feature tensor. For each of the sample transaction sequences, the masked sample feature tensor is input into the feature encoder to be trained to obtain the sample output vector of each sample transaction record in the sample transaction sequence. Based on the output feature label of the masked feature in the sample output vector and the original label of the masked feature in the sample feature tensor, the first loss value corresponding to the first loss function is determined. For each sample transaction sequence, the context representation vector of each sample transaction record in the sample transaction sequence is sequentially input into the sequence decoder to be trained according to the transaction occurrence time of each sample transaction record. The sequence decoder to be trained analyzes the temporal correlation between multiple context representation vectors to obtain the predicted feature vector of the predicted transaction record based on the temporal correlation. The context representation vector of the sample transaction record is the feature representation located at a preset position in the sample output vector of the sample transaction record. For each of the sample transaction sequences, a second loss value corresponding to the second loss function is determined based on the predicted feature vector of the predicted transaction record and the sample feature vector of the sample transaction record corresponding to the predicted transaction record. Based on the first loss value corresponding to the first loss function and the second loss value corresponding to the second loss function, determine the target loss value corresponding to the joint loss function; The parameters of the feature encoder and the sequence decoder to be trained are iteratively optimized based on the target loss value corresponding to the joint loss function until the target loss value meets the preset convergence condition.

8. A time-series data processing apparatus, characterized in that, The device includes: The transaction feature processing module is used to vectorize each transaction feature in each transaction record in the target transaction sequence to obtain a feature vector sequence for each transaction record, wherein the feature vector sequence includes the feature vectors corresponding to each transaction feature. The feature tensor generation module is used to generate the target feature tensor of the target transaction sequence based on the feature vector sequence of each transaction record and the transaction occurrence time of each transaction record. The feature vector encoding module is used to input the target feature tensor into the feature encoder, and to encode the correlation between each feature vector in the feature vector sequence of each transaction record to obtain the comprehensive feature vector of each transaction record. The memory vector processing module is used to input the comprehensive feature vector of each transaction record into the sequence decoder in sequence according to the transaction occurrence time of each transaction record. The sequence decoder analyzes the temporal correlation between multiple comprehensive feature vectors to obtain the context memory vector of the target transaction sequence at the target time step based on the temporal correlation. The context memory vector is used to perform a classification task on the target transaction sequence.

9. A time-series data processing device, characterized in that, include: Processor and memory storing computer program instructions; When the processor executes computer program instructions, it implements the timing data processing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the timing data processing method as described in any one of claims 1-7.

11. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the timing data processing method according to any one of claims 1-7.

Images