Ethereum account continuous learning method based on replay memory and distillation constraint

Abstract

The invention relates to technologies of block chains, deep learning and the like, and discloses an Ethereum account continuous learning method based on replay memory and distillation constraint. According to the method, a transaction sequence is constructed by using block chain transaction data, self-supervised pre-training is performed by using a high and low frequency fusion Transform model, and account embedding representation is extracted. In the continuous learning stage, a representation diversity replay buffer area is designed, a representative sample is selected according to the clustering result of the account embedding characteristics, and the sampling weight is determined in combination with the account activeness and the time span; meanwhile, an index moving average teacher model is introduced to realize distillation constraint, old knowledge representation consistency is kept, key parameter changes are limited through elastic weight regularization, and disastrous forgetting is prevented. According to the method, incremental adaptive learning of Ethereum account representation is realized, the model retraining cost is remarkably reduced, the stability and continuous learning ability of account representation are improved, and the method is suitable for intelligent analysis tasks in a dynamic block chain environment.

Description

Technical Field

[0001] This invention belongs to the field of blockchain intelligent analysis and machine learning technology, specifically involving technologies such as blockchain, deep learning, Transformer, and continuous learning, and particularly application scenarios such as Ethereum account representation learning, phishing identification, account similarity analysis, and on-chain risk monitoring. Background Technology

[0004] In existing technologies, account behavior recognition and representation learning primarily rely on centralized pre-trained models based on full transaction data. As new transaction data is continuously generated, a large-scale retraining process needs to be performed, which not only consumes enormous computational resources but also leads to the "catastrophic forgetting" problem, resulting in the loss of old knowledge and a degradation in the ability to recognize historical patterns. Furthermore, Ethereum account behavior patterns exhibit significant diversity and long-tail distribution characteristics. Traditional continuous learning methods based on fixed sample replay often suffer from insufficient representativeness and information redundancy in sample selection, making it difficult to maintain representation diversity while improving model generalization ability.

[0005] To address the issues of poor representation learning stability, catastrophic forgetting, and over-smoothing of features in Ethereum accounts under dynamic incremental data scenarios, this invention introduces a high-low frequency fusion mechanism into the traditional Transformer structure. This allows the model to simultaneously capture the local high-frequency dynamics and global low-frequency trends of transaction behavior, thereby improving feature stability and memory retention during incremental training. However, this method still relies on re-pre-training with the entire dataset, lacking adaptive learning capabilities for incremental data and prone to knowledge forgetting. Building upon this framework, this invention further proposes a continuous learning training method combining replay memory and distillation constraints. By introducing a representation diversity replay buffer, an exponential moving average (EMA) distillation mechanism, and elastic weight consolidation (EWC), it achieves representation stability and knowledge retention during incremental training.

[0006] Therefore, there is an urgent need for a method that can continuously update and stably learn account embedding representations through an efficient replay memory management mechanism and knowledge distillation constraint strategy, while Ethereum transaction data is constantly being updated. This would ensure the ability to learn new knowledge while effectively preventing the forgetting of old knowledge, and improve the long-term adaptability and robustness of the model in on-chain account recognition and behavior analysis tasks. Summary of the Invention

[0007] Existing Ethereum account representation learning methods rely on retraining with the entire dataset, resulting in high computational costs and susceptibility to catastrophic forgetting, making it difficult to maintain knowledge continuity under dynamic incremental data. Furthermore, fixed-sample replay strategies lack representativeness, making it difficult to balance model stability and representation diversity. This invention aims to address these issues by proposing a continuous learning training method for Ethereum accounts based on replay memory and distillation constraints. This method enhances the model's adaptability to incremental data, effectively prevents the forgetting of old knowledge, and improves the stability and continuous learning performance of account representations.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] S1) Obtain transaction data;

[0010] S2) Construct transaction sequence data based on transaction data;

[0011] S3) Use transaction sequence data to perform self-supervised pre-training on the joint framework of mask address prediction and contrastive learning based on high- and low-frequency fusion Transformer; and obtain the Transformer encoder with updated parameters.

[0012] S4) Construct a replay buffer to represent diversity and store historical account samples;

[0013] S5) When new transaction data arrives, select a portion of old samples from the replay buffer according to the sampling weight, and mix them with the new samples to form an incremental training batch.

[0014] S6) Introduce the EMA teacher model for distillation constraints during the training process;

[0015] S7) Calculate the importance of model parameters based on the Fisher information matrix and use EWC regularization to constrain the range of key parameter variations;

[0016] S8) Jointly optimize the mask prediction loss, contrastive learning loss, distillation loss and EWC regularization loss to obtain the updated account representation model.

[0017] Specifically, step S1) includes: obtaining transaction data from the Ethereum browser open interface, and using the Ethereum system node data cleaning component to extract structured transaction table data from the raw transaction data;

[0018] Further, step S2) includes:

[0019] S21) For an externally owned account with A0, select all transactions that are initiated or received by it, and sort the transactions in descending order according to the timestamp to form the transaction sequence of A0; for each transaction, there are eight features: address, account type, timestamp, amount, input / output type, number of transactions, location, and transaction duration. Among them, the input / output type indicates whether the transaction is received or initiated by A0, and the account type indicates whether the account is an externally owned account or a contract.

[0020] S22) For each transaction sequence, a virtual self-transaction is inserted at the beginning of the sequence. Its address feature is set to A0, the number of transactions is set to the total number of transactions in the sequence, the amount and input / output type are set to the total amount of the sequence flowing in / out, and all other features are set to empty. The virtual self-transaction is used to represent the owner of the transaction sequence.

[0021] S23) Aggregate consecutive repeating transactions that have the same address, the same input / output type, and are initiated by the same initiator within a certain period of time. Aggregate consecutive repeating transactions by merging their transaction amounts and transaction counts. The timestamp of the aggregated transaction is set to the first timestamp of the original transaction, and the transaction duration is set to the difference between the last timestamp and the first timestamp of the original transaction.

[0022] Further, in step S3), the pre-training process of the joint framework for mask address prediction and contrastive learning based on high- and low-frequency fusion Transformer is as follows:

[0023] S31) Input the constructed transaction sequence data into the jointly constructed mask address prediction and contrast learning framework based on high and low frequency fusion Transformer;

[0024] S32) Determine the hyperparameters of the framework, select the training optimizer and loss function, and then start training;

[0025] S33) Determine whether the training has reached the stopping condition;

[0026] (S34) If yes, end the pre-training and obtain the trained Transformer encoder; otherwise, update the parameters of the Transformer encoder using the backpropagation algorithm and return to step S33.

[0027] Furthermore, the high-low frequency fusion Transformer model described in step S3) includes an L-layer high-low frequency fusion Transformer layer, each layer consisting of a high-low frequency information attention fusion sublayer and a position feedforward sublayer; the high-low frequency information attention fusion sublayer combines a self-attention matrix and a filter matrix adjusted by Fourier transform to alleviate the over-smoothing problem caused by the low-pass filtering characteristics of self-attention; the calculation process of the high-low frequency fusion Transformer layer is described by the following formula:

[0028] S l =α(IFFT(FFT(X) l )☉M low )++βIFFT(FFT(X l )☉M high ))+(1-α)A l X l

[0029] Among them, S l X is the output of the high- and low-frequency information attention fusion sublayer of layer l. l This is the input of the l-th layer, α is the balance factor, FFT is the Fast Fourier Transform, IFFT is the Inverse Fourier Transform, and M... low M high This is a mask used for frequency partitioning, ⊙ is element-wise multiplication, β is a learnable weight parameter, and A... l It is a self-attention matrix.

[0030]

[0031] in, It is the output after multi-head self-attention operation, S h It is the h-th head output, W O It is a learnable projection matrix.

[0032]

[0033] Where GELU is the activation function of the Gaussian error linear unit. It is a learnable weight matrix. It is a learnable bias parameter. The input is fed into a feedforward network, which performs a nonlinear mapping through two layers of linear transformation and activation functions.

[0034]

[0035] Among them, X l+1 It is the input of the (l+1)th layer, which is processed by residual connection, Dropout, and layer normalization to obtain the input of the next layer.

[0036] Further, in step S3), during the pre-training process of the mask prediction task based on the high-low frequency fusion Transformer and the joint framework of contrastive learning, the transaction address features are randomly masked and their true values ​​are predicted. The contrastive loss between the positive sample address (mask address) and the randomly sampled negative sample address is calculated as the objective function of the pre-training. The optimization objective is to minimize the distance with the positive sample address and maximize the distance with the negative sample address. The calculation process is described in the following formula:

[0037]

[0038] Where M is the set of mask addresses in the sequence, x m It is the encoded unmasked context information N is a set of negative samples randomly sampled from each transaction sequence, a p It is the address embedding of its mask address, a n It is the address embedding from a negative sample address of N, and τ is the temperature coefficient.

[0039] Further, in step S3), during the pre-training process of the contrastive learning task of the joint framework for mask address prediction and contrastive learning based on high- and low-frequency fusion Transformer, the transaction sequences of the same account are randomly augmented (e.g., randomly pruned, time-perturbed, etc.), and the InfoNCE Loss function is used for contrastive learning to minimize the distance between positive sample embeddings and maximize the distance between negative samples; the calculation process is described by the following formula:

[0040]

[0041] Where τ is the temperature coefficient, q is the encoded representation vector of the query sample, and k + It is the encoding vector of the positive sample corresponding to the query sample, k i It is the encoding vector of the negative sample that does not correspond to the query sample.

[0042] Further, in step S4), the process of representing diversity replay buffer is as follows:

[0043] S41) Perform cluster analysis on the embedding vectors of historical account samples, preferably using MiniBatchKMeans or other Euclidean distance-based clustering algorithms to divide the account samples into multiple clusters, each cluster corresponding to a typical transaction behavior pattern.

[0044] S42) In each cluster, the sample closest to the cluster center is selected as the "cluster center sample" to represent the mainstream behavioral characteristics of the cluster; at the same time, the sample farthest from the cluster center is selected as the "outlier sample" to supplement a few abnormal or boundary behavioral patterns, thereby ensuring the diversity of sample distribution.

[0045] S43) Calculate the activity level a for each account sample. i Transaction time span t i and embedding representation diversity d i ;

[0046] Among them, account activity level a i Calculated from the number of transactions per unit time; transaction time span t i The normalized value of the difference between the earliest and latest times of all transactions in the account; embedding representation of diversity d i The formula used to reflect the degree of difference between samples in the embedding space is as follows:

[0047]

[0048] Among them, e i It is the high-dimensional embedding vector of the account, c k It is the center sample vector of the cluster to which the account belongs, μ k It is the average Euclidean distance from the account embedding to the central sample within the current cluster.

[0049] S44) The sampling weights of samples are determined based on activity, time span, and embedding diversity. The weighted calculation formula is as follows:

[0050] w i =β1·a i +β2·t i +(1-β1-β2)·d i

[0051] Among them, β1 and β2 are balance coefficients used to control the proportion of influence of different factors in the sampling process.

[0052] S45) Select representative and diverse account samples from the historical sample set according to the sampling weights, construct a replay buffer, and periodically recalculate the sample embedding and clustering results according to the model update in order to maintain the dynamic diversity and distribution balance of the buffer samples.

[0053] Further, step S6) includes: the EMA teacher model distillation constraint process includes the following steps:

[0054] S61) In the incremental training phase, let the current model to be trained be the Student Model, with parameters θ.student Simultaneously, maintain a teacher model with the same structure, whose parameters θ teacher The student model parameters are updated using the EMA method, and the update formula is:

[0055] θ teacher ←τ·θ teacher +(1-τ)·θ student

[0056] Where τ∈[0,1) is the smoothing coefficient, which controls the update speed of the teacher model parameters; the larger τ is, the smoother the change of the teacher model parameters, thus retaining more historical knowledge representation.

[0057] (S62) In each training round, the student model receives an incremental training batch consisting of a mixture of new samples and replay samples, and generates a predictive embedding representation h through forward propagation. s The teacher model uses the same input data to generate a stable embedding representation h. t .

[0058] S63), define the distillation constraint loss function L distill This is used to minimize the difference between the output embeddings of the student model and the teacher model; preferably, it can take one of the following two forms:

[0059] Similarity-based distillation:

[0060] L distill =1-cos(h) s ,h t )

[0061] Where cos(·) represents cosine similarity, the smaller the loss value, the closer the student model output is to the teacher model;

[0062] Based on the form of distribution alignment:

[0063]

[0064] in This represents the L2 norm distance, used to measure the Euclidean difference between the embedding spaces of two models.

[0065] S64) After each iteration, the current parameters of the student model are updated to the teacher model according to the EMA rule to achieve dynamic and smooth updates of the teacher model; this mechanism enables the teacher model to form a "moving average representation of historical knowledge" in the parameter space, avoiding drastic drift of the model during incremental data training.

[0066] S65) In the overall optimization objective, the distillation constraint loss L distill With mask prediction loss LMAP Contrast learning loss L InfoNCE EWC regularization loss L EWC Joint optimization is performed, and the overall loss function is defined as:

[0067] L = L MAP +λ1L InfoNCE +λ2L distill +λ3L EWC

[0068] Among them, λ1, λ2, and λ3 are balance coefficients used to control the weight ratio of different loss terms.

[0069] Further, step S7) includes: the EWC constraint process based on the Fisher information matrix includes the following steps:

[0070] S71) After completing each stage of model training, calculate the sensitivity of the current model parameters to task performance to determine the importance of each parameter. Preferably, Fisher's information matrix is ​​used for quantification, and the importance index F of the i-th parameter is... i Defined as:

[0071]

[0072] Where p(y|x, θ) is the output probability distribution of the model under parameter θ. This represents the loss function with respect to the parameter θ. i gradient, This is the expectation operation. The larger this value, the more significant the impact of this parameter on model performance, and it needs to be carefully protected in subsequent incremental learning.

[0073] S72) When new data arrives and incremental training is performed, to prevent drastic drift in key parameters, EWC regularization constraints are applied to the model parameters. The constraint loss is defined as:

[0074]

[0075] Where, θ i These are the current model parameters. It is the historically optimal parameter, F i λ represents the parameter importance in the Fisher information matrix estimation, and λ is the regularization coefficient.

[0076] The EWC mechanism (S73) imposes strong constraints on highly important parameters during parameter updates to prevent them from deviating from their historical optimal values; while less important parameters are allowed to adjust freely under new data to learn new knowledge. This mechanism ensures the model's adaptability to new tasks while maintaining its memory of knowledge from old tasks, effectively mitigating the problem of catastrophic forgetting.

[0077] The beneficial effects of this invention are as follows:

[0078] 1) The high-low frequency fusion Transformer introduces a dual masking mechanism in the frequency domain, enabling the model to maintain its time-dependent modeling capabilities while also taking into account the multi-scale feature representation of the transaction sequence.

[0079] 2) By combining replay memory with distillation constraint mechanism, the model can be continuously updated when receiving new transaction data without re-pre-training, thus achieving continuous learning capability;

[0080] 3) Introducing EMA distillation and EWC regularization mechanisms effectively maintains the embedding stability of old task knowledge and prevents knowledge forgetting;

[0081] 4) By selecting cluster centers and outliers through clustering, and combining activity, time span and diversity to calculate sampling weights, the representativeness and balance of replay samples are ensured, and the representativeness and diversity of samples are enhanced.

[0082] 5) This invention can adapt to the dynamic changes in Ethereum transaction behavior, effectively improve the generalization ability of account embedding and the accuracy of risk identification, and enhance the robustness of the model. Attached Figure Description

[0083] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0084] Figure 2 This is a schematic diagram of the model framework of the present invention. Detailed Implementation

[0085] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] This invention proposes a continuous learning training method for Ethereum accounts based on replay memory and distillation constraints. The main process of the method is as follows: Figure 1 As shown. The specific implementation process is as follows:

[0087] S1) Obtain transaction data;

[0088] S2) Construct transaction sequence data based on transaction data;

[0089] S3) Use transaction sequence data to perform self-supervised pre-training on the joint framework of mask address prediction and contrastive learning based on high- and low-frequency fusion Transformer; and obtain the Transformer encoder with updated parameters.

[0090] S4) Construct a replay buffer to represent diversity and store historical account samples;

[0091] S5) When new transaction data arrives, select a portion of old samples from the replay buffer according to the sampling weight, and mix them with the new samples to form an incremental training batch.

[0092] S6) Introduce the EMA teacher model for distillation constraints during the training process;

[0093] S7) Calculate the importance of model parameters based on the Fisher information matrix and use EWC regularization to constrain the range of key parameter variations;

[0094] S8) Jointly optimize the mask prediction loss, contrastive learning loss, distillation loss and EWC regularization loss to obtain the updated account representation model.

[0095] Specifically, step S1) includes: obtaining transaction data from the Ethereum browser open interface, and using the Ethereum system node data cleaning component to extract structured transaction table data from the raw transaction data;

[0096] Further, step S2) includes:

[0097] S21) For an externally owned account with A0, select all transactions that are initiated or received by it, and sort the transactions in descending order according to the timestamp to form the transaction sequence of A0; for each transaction, there are eight features: address, account type, timestamp, amount, input / output type, number of transactions, location, and transaction duration. Among them, the input / output type indicates whether the transaction is received or initiated by A0, and the account type indicates whether the account is an externally owned account or a contract.

[0098] S22) For each transaction sequence, a virtual self-transaction is inserted at the beginning of the sequence. Its address feature is set to A0, the number of transactions is set to the total number of transactions in the sequence, the amount and input / output type are set to the total amount of the sequence flowing in / out, and all other features are set to empty. The virtual self-transaction is used to represent the owner of the transaction sequence.

[0099] S23) Aggregate consecutive repeating transactions that have the same address, the same input / output type, and are initiated by the same initiator within a certain period of time. Aggregate consecutive repeating transactions by merging their transaction amounts and transaction counts. The timestamp of the aggregated transaction is set to the first timestamp of the original transaction, and the transaction duration is set to the difference between the last timestamp and the first timestamp of the original transaction.

[0100] Further, in step S3), the pre-training process of the joint framework for mask address prediction and contrastive learning based on high- and low-frequency fusion Transformer is as follows:

[0101] S31) Input the constructed transaction sequence data into the jointly constructed mask address prediction and contrast learning framework based on high and low frequency fusion Transformer;

[0102] S32) Determine the hyperparameters of the framework, select the training optimizer and loss function, and then start training;

[0103] S33) Determine whether the training has reached the stopping condition;

[0104] (S34) If yes, end the pre-training and obtain the trained Transformer encoder; otherwise, update the parameters of the Transformer encoder using the backpropagation algorithm and return to step S33.

[0105] Furthermore, the high-low frequency fusion Transformer model described in step S3) includes an L-layer high-low frequency fusion Transformer layer, each layer consisting of a high-low frequency information attention fusion sublayer and a position feedforward sublayer; the high-low frequency information attention fusion sublayer combines a self-attention matrix and a filter matrix adjusted by Fourier transform to alleviate the over-smoothing problem caused by the low-pass filtering characteristics of self-attention; the calculation process of the high-low frequency fusion Transformer layer is described by the following formula:

[0106] S l =α(IFFT(FFT(X) l )⊙M low )+βIFFT(FFT(X l )⊙M high ))+(1-α)A l X l

[0107] Among them, S l X is the output of the high- and low-frequency information attention fusion sublayer of layer l. l This is the input of the l-th layer, α is the balance factor, FFT is the Fast Fourier Transform, IFFT is the Inverse Fourier Transform, and M...low M high This is a mask used for frequency partitioning, ⊙ is element-wise multiplication, β is a learnable weight parameter, and A... l It is a self-attention matrix.

[0108]

[0109] in, It is the output after multi-head self-attention operation, S h It is the h-th head output, W O It is a learnable projection matrix.

[0110]

[0111] Where GELU is the activation function of the Gaussian error linear unit. It is a learnable weight matrix. It is a learnable bias parameter. The input is fed into a feedforward network, which performs a nonlinear mapping through two layers of linear transformation and activation functions.

[0112]

[0113] Among them, X l+1 It is the input of the (l+1)th layer, which is processed by residual connection, Dropout, and layer normalization to obtain the input of the next layer.

[0114] Further, in step S3), during the pre-training process of the mask prediction task based on the high-low frequency fusion Transformer and the joint framework of contrastive learning, the transaction address features are randomly masked and their true values ​​are predicted. The contrastive loss between the positive sample address (mask address) and the randomly sampled negative sample address is calculated as the objective function of the pre-training. The optimization objective is to minimize the distance with the positive sample address and maximize the distance with the negative sample address. The calculation process is described in the following formula:

[0115]

[0116] Where M is the set of mask addresses in the sequence, x m It is the encoded unmasked context information N is a set of negative samples randomly sampled from each transaction sequence, a p It is the address embedding of its mask address, a n It is the address embedding from a negative sample address of N, and τ is the temperature coefficient.

[0117] Further, in step S3), during the pre-training process of the contrastive learning task of the joint framework for mask address prediction and contrastive learning based on high- and low-frequency fusion Transformer, the transaction sequences of the same account are randomly augmented (e.g., randomly pruned, time-perturbed, etc.), and the InfoNCE Loss function is used for contrastive learning to minimize the distance between positive sample embeddings and maximize the distance between negative samples; the calculation process is described by the following formula:

[0118]

[0119] Where τ is the temperature coefficient, q is the encoded representation vector of the query sample, and k + It is the encoding vector of the positive sample corresponding to the query sample, k i It is the encoding vector of the negative sample that does not correspond to the query sample.

[0120] Further, in step S4), the process of representing diversity replay buffer is as follows:

[0121] S41) Perform cluster analysis on the embedding vectors of historical account samples, preferably using MiniBatchKMeans or other Euclidean distance-based clustering algorithms to divide the account samples into multiple clusters, each cluster corresponding to a typical transaction behavior pattern.

[0122] S42) In each cluster, the sample closest to the cluster center is selected as the "cluster center sample" to represent the mainstream behavioral characteristics of the cluster; at the same time, the sample farthest from the cluster center is selected as the "outlier sample" to supplement a few abnormal or boundary behavioral patterns, thereby ensuring the diversity of sample distribution.

[0123] S43) Calculate the activity level a for each account sample. i Transaction time span t i and embedding representation diversity d i ;

[0124] Among them, account activity level a i Calculated from the number of transactions per unit time; transaction time span t i The normalized value of the difference between the earliest and latest times of all transactions in the account; embedding representation of diversity d i The formula used to reflect the degree of difference between samples in the embedding space is as follows:

[0125]

[0126] Among them, e i It is the high-dimensional embedding vector of the account, c k It is the center sample vector of the cluster to which the account belongs, μ kIt is the average Euclidean distance from the account embedding to the central sample within the current cluster.

[0127] S44) The sampling weights of samples are determined based on activity, time span, and embedding diversity. The weighted calculation formula is as follows:

[0128] w i =β1·a i +β2·t i +(1-β1-β2)·d i

[0129] Among them, β1 and β2 are balance coefficients used to control the proportion of influence of different factors in the sampling process.

[0130] S45) Select representative and diverse account samples from the historical sample set according to the sampling weights, construct a replay buffer, and periodically recalculate the sample embedding and clustering results according to the model update in order to maintain the dynamic diversity and distribution balance of the buffer samples.

[0131] Further, step S6) includes: the EMA teacher model distillation constraint process includes the following steps:

[0132] S61) In the incremental training phase, let the current model to be trained be the Student Model, with parameters θ. student Simultaneously, maintain a teacher model with the same structure, whose parameters θ teacher The student model parameters are updated using the EMA method, and the update formula is:

[0133] θ teacher ←τ·θ teacher +(1-τ)·θ student

[0134] Where r∈[0,1) is the smoothing coefficient, which controls the update speed of the teacher model parameters; the larger τ is, the smoother the change of the teacher model parameters, thus retaining more historical knowledge representation.

[0135] (S62) In each training round, the student model receives an incremental training batch consisting of a mixture of new samples and replay samples, and generates a predictive embedding representation h through forward propagation. s The teacher model uses the same input data to generate a stable embedding representation h. t .

[0136] S63), define the distillation constraint loss function L distill This is used to minimize the difference between the output embeddings of the student model and the teacher model; preferably, it can take one of the following two forms:

[0137] Similarity-based distillation:

[0138] L distill =1-cos(h) s ,h t )

[0139] Where cos(·) represents cosine similarity, the smaller the loss value, the closer the student model output is to the teacher model;

[0140] Based on the form of distribution alignment:

[0141]

[0142] in This represents the L2 norm distance, used to measure the Euclidean difference between the embedding spaces of two models.

[0143] S64) After each iteration, the current parameters of the student model are updated to the teacher model according to the EMA rule to achieve dynamic and smooth updates of the teacher model; this mechanism enables the teacher model to form a "moving average representation of historical knowledge" in the parameter space, avoiding drastic drift of the model during incremental data training.

[0144] S65) In the overall optimization objective, the distillation constraint loss L distill With mask prediction loss L MAP Contrast learning loss L InfoNCE EWC regularization loss L EWC Joint optimization is performed, and the overall loss function is defined as:

[0145] L = L MAP +λ1L InfoNCE +λ2L distill +λ3L EWC

[0146] Among them, λ1, λ2, and λ3 are balance coefficients used to control the weight ratio of different loss terms.

[0147] Further, step S7) includes: the EWC constraint process based on the Fisher information matrix includes the following steps:

[0148] S71) After completing each stage of model training, calculate the sensitivity of the current model parameters to task performance to determine the importance of each parameter. Preferably, Fisher's information matrix is ​​used for quantification, and the importance index F of the i-th parameter is... i Defined as:

[0149]

[0150] Where p(y|x, θ) is the output probability distribution of the model under parameter θ. This represents the loss function with respect to the parameter θ. i gradient, This is the expectation operation. The larger this value, the more significant the impact of this parameter on model performance, and it needs to be carefully protected in subsequent incremental learning.

[0151] S72) When new data arrives and incremental training is performed, to prevent drastic drift in key parameters, EWC regularization constraints are applied to the model parameters. The constraint loss is defined as:

[0152]

[0153] Where, θ i These are the current model parameters. It is the historically optimal parameter, E i λ represents the parameter importance in the Fisher information matrix estimation, and λ is the regularization coefficient.

[0154] The EWC mechanism (S73) imposes strong constraints on highly important parameters during parameter updates to prevent them from deviating from their historical optimal values; while less important parameters are allowed to adjust freely under new data to learn new knowledge. This mechanism ensures the model's adaptability to new tasks while maintaining its memory of knowledge from old tasks, effectively mitigating the problem of catastrophic forgetting.

[0155] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for continuous learning of Ethereum accounts based on replay memory and distillation constraints, characterized in that: Includes the following steps: S1) Obtain transaction data; S2) Construct transaction sequence data based on the transaction data; S3) Use the transaction sequence data to perform self-supervised pre-training on the joint framework of mask address prediction and contrastive learning based on the high-low frequency fusion Transformer model to obtain account embedding representation; S4) Establish a representation diversity replay buffer to dynamically maintain historical account samples. The replay buffer selects representative samples based on the clustering results of account embedding features and determines sampling weights based on account activity, transaction time span, and embedding representation diversity. S5) When new transaction data arrives, old samples are sampled from the replay buffer and mixed with new samples to form a training batch. S6) During training, an exponential moving average teacher model is introduced for distillation constraints to ensure that the account embedding distribution output by the current model is consistent with the teacher model. S7) The importance of model parameters is evaluated based on the Fisher information matrix, and constraints are imposed on key parameters through elastic weight regularization to reduce catastrophic forgetting. S8) The mask prediction loss, contrastive learning loss, distillation loss, and regularization loss are combined according to preset weights to form a comprehensive loss function, and the model parameters are updated based on the comprehensive loss to achieve continuous learning of Ethereum account representations.

2. The method according to claim 1, characterized in that, Step S3) includes: The Transformer encoder includes multiple Transformer layers, each Transformer layer containing a high- and low-frequency information attention fusion sublayer and a position feedforward sublayer; wherein the high- and low-frequency information attention fusion sublayer combines a self-attention matrix and a filter matrix adjusted by Fourier transform to alleviate the over-smoothing problem caused by the low-pass filtering characteristics of self-attention.

3. The method according to claim 1, characterized in that, Step S4) includes: The replay buffer characterizes the diversity by performing cluster analysis on the account embedding features, selecting the center sample of each cluster and the outlier sample with the largest distance from the center for replay, so as to maintain the representativeness and diversity of the data at the same time.

4. The method according to claim 2, characterized in that, The calculation formula for the high- and low-frequency information attention fusion sublayer is as follows: S l =α[IFFT(FFT(X l )☉M low )+βIFFT(FFT(X l )☉M high )]X l +(1-α)A l X l Among them, X l S is the input of the l-th layer. l The output of the high- and low-frequency information attention fusion sublayer of layer l is given, where α is the balance factor, FFT is the Fast Fourier Transform, IFFT is the Inverse Fourier Transform, and M is the value of M. low M high Here, ⊙ represents the mask used for frequency partitioning, β represents element-level multiplication, and A represents the learnable weight parameters. l This is the self-attention matrix.

5. The method according to claim 1, characterized in that, The high- and low-frequency fusion Transformer model adopts a joint self-supervised strategy of mask prediction and contrastive learning. In the mask prediction task, transaction addresses are randomly masked and prediction errors are minimized. In the contrastive learning task, the embedding distance is minimized for different enhanced samples of the same account transaction sequence, so as to improve the discriminativeness and temporal robustness of the embedded features.

6. The method according to claim 1, characterized in that, The sampling weights are determined based on account activity, transaction time span, and embedding representation diversity, and are calculated using the following formula: w i =β1·a i +β2·t i +(1-β1-β2)·d i Among them, a i To measure account transaction activity, t i The normalized value for the transaction time span, β1 and β2 are the balance coefficients in the interval [0,1], and d i To represent diversity in the embedding space, reflecting the degree of difference of the account in the embedding space; Preferably, the d i It can be calculated using the following formula: Among them, e i c is the high-dimensional embedding vector of the account. k μ is the center sample vector of the cluster to which this account belongs. k This represents the average Euclidean distance between the account embeddings within the current cluster and the central sample.

7. The method according to claim 1, characterized in that, Step S6) includes: updating the teacher model parameters using an exponential moving average of the current model parameters, with the following update formula: i teocher ←t·i teacher +(1-τ)·θ student Where, θ teacher For the teacher model parameters, θ student τ represents the current model parameters, and τ is the smoothing coefficient in the interval [0, 1].

8. The method according to claim 1, characterized in that, Step S7) includes: The penalty term for elastic weight regularization is defined as: Where, θ i These are the current model parameters. F is the historically optimal parameter. i The parameter importance is given by Fisher information matrix estimation, where λ is the regularization coefficient.

Images