User click behavior sequence prediction method and system based on Mamba and data enhancement

Abstract

The present invention provides a method for predicting user click behavior sequence based on Mamba and data enhancement, including: collecting information of users, items, item categories, and user click behavior sequence, performing data cleaning, initializing representation vectors of users, items, categories, and candidate set items; defining and designing a Mamba module suitable for user click behavior sequence, using the selective propagation and forgetting mechanism of the Mamba module to perform dynamic parameter adjustment, preprocessing the user click behavior sequence and extracting behavior features; defining and designing a data enhancement method to generate diversified data samples; integrating the results of Mamba preprocessing and data enhancement, using masking technology in the training process to train only partial data of the non-filled part that can represent the click behavior in the corresponding batch; optimizing parameters according to a loss function, and evaluating the performance of the method on a validation set. The present invention improves the efficiency of processing long sequence information without reducing the prediction accuracy, relies on data enhancement technology to perform diversified processing on the user click behavior sequence, further captures the user's potential diversified interests, and realizes efficient prediction of the user's click behavior.

Description

Technical Field

[0001] The present invention relates to the field of user behavior prediction and data enhancement, and more specifically to a method for predicting a user click behavior sequence. Background Art

[0002] Mamba is a new variant of the Selective State Space Models type, designed to address computational efficiency and content reasoning issues in modeling long sequences of information. Mamba uses hardware-aware parallel algorithms for efficient computation, dynamically adjusts parameters based on current input, and selectively propagates or forgets information.

[0003] Data augmentation technology generates diverse data samples and increases the amount of training data, thereby improving the generalization ability of prediction results. It is particularly effective in dealing with the overfitting problem of a single type of data.

[0004] The user click behavior sequence is usually represented by D = {U, G, C, S, V} and the candidate set H, where U represents the user entity, G represents the item entity, C represents the item category entity, S represents the user click history behavior sequence, V represents the context entity, i.e., the context information related to the user click behavior (such as time, location, device used, etc.), and H represents the candidate set entity, i.e., the set of candidate items that the user may click. The specific task of user click behavior sequence analysis is to predict the number of items that a given user u clicked in the previous k times: g1, g2, ..., g k After that, the candidate item h is clicked for the k+1th time k+1 In the field of behavioral sequence modeling for click-through rate prediction, user click behavior sequence data usually contains multiple interests. The diversity of these interests not only increases the complexity of modeling, but a single modeling method may also cause the results to overfit to a single interest. These challenges can be effectively alleviated by diversifying the user click behavior sequence through data augmentation technology. However, user click behavior sequences are often very long, and traditional methods based on sequence models (such as RNN and Transformer) are inefficient in processing long sequence data, which easily leads to a waste of computing resources. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to efficiently process long sequence information without reducing the prediction accuracy and achieve efficient prediction of user click behavior.

[0006] The present invention solves the above technical problems through the following technical means: 1. A user click behavior sequence prediction method based on Mamba and data enhancement, characterized in that it includes the following steps:

[0007] Step S1: Collect information about users, items, item categories, and user click behavior sequences, perform data cleaning, format the data into a unified form, and divide the corresponding fields. Secondly, split the data set into training set, validation set, and test set, and initialize the representation vectors of users, items, categories, and candidate set items.

[0008] Step S2: define and design a Mamba module suitable for the user click behavior sequence, use the selective propagation and forgetting mechanism of the Mamba module to perform dynamic parameter adjustment, preprocess the user click behavior sequence and extract behavior features;

[0009] Step S3: define and design data enhancement methods to generate diverse data samples and increase the amount of training data;

[0010] Step S4: Integrate the results of Mamba preprocessing and data augmentation, and use masking technology in the training process to train only the non-filled part of the data that can represent the click behavior in the corresponding batch;

[0011] Step S5: Calculate the error using a suitable loss function, optimize the parameters based on the loss function, evaluate the performance of the method on the validation set, adjust the parameters to optimize the performance of the method, and finally, evaluate the performance of the method using the test set.

[0012] Furthermore, step S1 specifically includes:

[0013] Step 101: Collect the user's click behavior records on different e-commerce platforms, including the user ID set U = {u1, u2, ..., u |U|}、Product ID set G={g1,g2,...,g |G|} and product category ID set C = {c1, c2, ..., c |C|}, construct a mapping relationship from product ID to category ID, and collect the list of products or product categories clicked by each user S = {g1, g2, ..., g k}, forming the user's historical click sequence, collecting user context information, including time, location, device used, etc., forming a context set V = {v1, v2, ..., v |V|};

[0014] Step 102: Arrange the data into a unified format, including at least the following fields: a prediction label indicating whether the user clicks on a certain product, a unique user ID that identifies the user, the product ID of the currently clicked product or the prediction candidate set, the product category ID of the currently clicked product or the prediction candidate set, a list of historical product IDs clicked by the user or a list of historical product category IDs, and a context classification field;

[0015] Step 103: Split the data set into a training set, a validation set, and a test set. The training set is to select the first k products and their categories from the user's historical click sequence as input features, and the last product and its category as a label. The validation set is used for intermediate evaluation and parameter adjustment. The test set retains the last click behavior of each user for testing, forming a candidate product set H.

[0016] Step 104: Initialize the representation vector and the user representation vector Product representation vector Category representation vector Context Representation Vector Candidate set product representation vector Where D represents the dimension of the representation vector. Then, each representation vector is stored in its own representation vector dictionary as the initial input for the subsequent part.

[0017] Further, step S2 includes:

[0018] Step 201: Based on the design concept of SSM in the Mamba module, a state space model suitable for user behavior sequences is defined, and the original user history click behavior sequence at time t is defined as The click behavior sequence combined with context information for semantic enhancement is: By hiding the state Mapped into the final representation vector sequence Where N is the length of the original behavior sequence:

[0019] h′(t)=A·h(t)+B·x(t)+α·C·x aug (t)

[0020] y(t)=E·h(t)

[0021] Here and is a learnable parameter matrix, α is a predefined parameter used to control the degree of semantic enhancement. In order to process discrete user behavior sequences rather than continuous functions, it needs to be discretized by step size Δ. The discretized UBSSM expression is as follows:

[0022]

[0023] y k =E·h k

[0024] in,

[0025]

[0026] By converting the parameters from continuous form (Δ, A, B, C, E) to discrete form The calculation is performed in a linear recursive manner;

[0027] Step 202: Based on the design concept of the feedforward network in the Mamba module, define and design a behavior-aware feedforward network to further capture the user's potential interests through the hidden layer H:

[0028] UBFFN(H)=Dice(H·W (1) +b (1) )·W (2) +b (2)

[0029] in, Use the Dice activation function to adaptively adjust the correction point according to the distribution of the input data, so that its value is set to the mean of the input;

[0030] Step 203: define and design a new Mamba module, combine UBSSM and UBFFN, and build a new Mamba module;

[0031] Step 204: Configure the Mamba module and embed it into an appropriate network structure, including a single-layer and multi-layer stacked structure, and retain the parallel algorithm of Mamba hardware awareness. The output of each module is used as the input of the next module, and finally the feature representation F preprocessed by Mamba is obtained. mamba ;

[0032] Step 205: Add residual connections between each layer of Mamba modules, that is, directly add the input to the output, and at the same time, perform layer normalization on the output of each layer.

[0033] Furthermore, in step 204, for relatively simple user behavior sequences, a single-layer Mamba module is used to embed the combined model of SSM and UBFFN into one layer, and for complex user behavior sequences, a multi-layer stacking structure is used, that is, a stacking of multiple Mamba modules.

[0034] Further, step S3 includes:

[0035] Step 301: Given the target item feature representation T and the feature representation H corresponding to the historical click behavior sequence, n}, construct the basic enhanced behavior feature representation:

[0036] F base =[T, H, TH, T⊙H]

[0037] Among them, ⊙ represents the element-by-element product;

[0038] Step 302: Generate Gaussian noise N~N(0, σ 2 ), random noise is added to the historical click sequence to form a perturbed sequence representation:

[0039] H perturbed =H+N;

[0040] Step 303: Combine the target item feature representation T and the perturbed sequence representation H perturbed Perform concatenation to generate perturbation feature representation:

[0041] F perturbed =[T,H perturbed , TH perturbed , T⊙H perturbed ];

[0042] Step 304: Merge the base enhancement features with the perturbation features to form the final enhanced input feature representation:

[0043] F augmented =F base +F perturbed ;

[0044] Step 305: Normalize the enhanced input features:

[0045] F normalized =LayerNorm(F augmented ).

[0046] Further, step S4 includes:

[0047] Step 401: Integrate comprehensive features and transform the Mamba pre-processed features F mamba And the feature F after data enhancement normalized Splice to form a comprehensive feature matrix:

[0048] F combined =[F mamba , F normalized ];

[0049] Step 402: Construct a mask matrix M, whose element m ij Indicates whether the jth position of the i-th sequence sample is a filling part. The part to be filled refers to the sample sequence that has no click behavior at the current position:

[0050]

[0051] Step 403: Apply masking technology. During the training process, apply the mask matrix M to train only the features of the non-filled part:

[0052] Fmasked =F combined ⊙M

[0053] where ⊙ represents the element-wise product.

[0054] Further, step S5 includes:

[0055] Step 501: Select appropriate main loss function and auxiliary loss function and

[0056]

[0057] in, Indicates that based on F combined The predicted value of Only indicates that based on F normalized The predicted value of

[0058] Step 502: training the method using stochastic gradient descent and Adam optimizer;

[0059] Step 503: After the training is completed, the validation set is used to further optimize the internal parameters of the Mamba module and the relevant feature representation parameters, and finally the test set is used to evaluate the quality of the method.

[0060] The present invention also provides a user click behavior sequence prediction system based on Mamba and data enhancement, comprising:

[0061] Information collection module: used to collect information about users, items, item categories, and user click behavior sequences, perform data cleaning, format the data into a unified form, and divide the data into corresponding fields. Secondly, the data set is split into training set, validation set, and test set, and the representation vectors of users, items, categories, and candidate set items are initialized.

[0062] Mamba module design module: used to define and design Mamba modules suitable for user click behavior sequences, use the selective propagation and forgetting mechanism of Mamba modules to adjust parameters dynamically, and perform preprocessing and behavior feature extraction on user click behavior sequences;

[0063] Data sample generation module: define and design data enhancement methods, generate diverse data samples, and increase the amount of training data;

[0064] The training module is used to integrate the results of Mamba preprocessing and data enhancement. During the training process, masking technology is used to train only the non-filled part of the data that can represent the click behavior in the corresponding batch;

[0065] The optimization module is used to calculate the error using a suitable loss function, optimize the parameters according to the loss function, evaluate the performance of the method on the validation set, adjust the parameters to optimize the method performance, and finally, evaluate the method performance using the test set.

[0066] Furthermore, the information collection module specifically includes:

[0067] The click behavior recording unit is used to collect the click behavior records of users on different e-commerce platforms, including the user ID set U = {u1, u2, ..., u |U|}、Product ID set G={g1,g2,...,g |G|} and product category ID set C = {c1, c2, ..., c |C|}, construct a mapping relationship from product ID to category ID, and collect the list of products or product categories clicked by each user S = {g1, g2, ..., g k}, forming the user's historical click sequence, collecting user context information, including time, location, device used, etc., forming a context set V = {v1, v2, ..., v |V|};

[0068] A formatting unit is used to organize the data into a unified format, which at least includes the following fields: a prediction label indicating whether a user clicks on a certain product, a unique user ID that identifies the user, a product ID currently being clicked or in a prediction candidate set, a product category ID currently being clicked or in a prediction candidate set, a list of historical product IDs clicked by the user or a list of historical product category IDs, and a context classification field;

[0069] The splitting unit is used to split the data set into a training set, a validation set, and a test set. The training set selects the first k products and their categories from the user's historical click sequence as input features, and the last product and its category as a label. The validation set is used for intermediate evaluation and parameter adjustment. The test set retains the last click behavior of each user for testing to form a candidate product set H.

[0070] Vector representation unit, used to initialize the representation vector, initialize the user representation vector Product representation vector Category representation vector Context Representation Vector Candidate set product representation vector Where D represents the dimension of the representation vector. Then, each representation vector is stored in its own representation vector dictionary as the initial input for the subsequent part.

[0071] Furthermore, the Mamba module design module includes:

[0072] State space model definition unit: used to define the state space model applicable to the user behavior sequence according to the design concept of SSM in the Mamba module, and define the original user history click behavior sequence at time t as The click behavior sequence combined with context information for semantic enhancement is: By hiding the state Mapped into the final representation vector sequence Where N is the length of the original behavior sequence:

[0073] h′(t)=A·h(t)+B·x(t)+α·C·x aug (t)

[0074] y(t)=E·h(t)

[0075] Here and is a learnable parameter matrix, α is a predefined parameter used to control the degree of semantic enhancement. In order to process discrete user behavior sequences rather than continuous functions, it needs to be discretized by step size Δ. The discretized UBSSM expression is as follows:

[0076]

[0077] y k =E·h k

[0078] in,

[0079]

[0080] By converting the parameters from continuous form (Δ, A, B, C, E) to discrete form The calculation is performed in a linear recursive manner;

[0081] Potential interest capture unit: It is used to define and design a behavior-aware feedforward network based on the design concept of the feedforward network in the Mamba module, and further capture the user's potential interests through the hidden layer H:

[0082] UBFFN(H)=Dice(H·W (1) +b (1) )·W (2) +b (2)

[0083] in, Use the Dice activation function to adaptively adjust the correction point according to the distribution of the input data, so that its value is set to the mean of the input;

[0084] New module building unit: used to define and design new Mamba modules, combining UBSSM and UBFFN to build new Mamba modules;

[0085] Configuration unit: used to configure the Mamba module and embed it into the appropriate network structure, including single-layer and multi-layer stacking structures, and retain the parallel algorithm of Mamba hardware awareness. The output of each module is used as the input of the next module, and finally the feature representation F preprocessed by Mamba is obtained. mamba ;

[0086] Connection unit: used to add residual connections between each layer of Mamba modules, that is, directly add the input to the output, and at the same time, perform layer normalization on the output of each layer;

[0087] The data sample generation module includes:

[0088] Feature representation unit: used to represent the feature representation T of the target item and the feature representation H corresponding to the historical click behavior sequence = {h1, h2, ..., h n}, construct the basic enhanced behavior feature representation:

[0089] F base =[T, H, TH, ToH]

[0090] Among them, ⊙ represents the element-by-element product;

[0091] Sequence generation unit: used to generate Gaussian noise N~N(0, σ 2 ), random noise is added to the historical click sequence to form a perturbed sequence representation:

[0092] H perturbed =H+N;

[0093] Concatenation unit: used to combine the target item feature representation T with the perturbed sequence representation H perturbed Perform concatenation to generate perturbation feature representation:

[0094] F perturbed =[T,H perturbed , TH perturbed , T⊙H perturbed ];

[0095] Merging unit: used to merge the basic enhanced features with the perturbation features to form the final enhanced input feature representation:

[0096] F augmented =F base +F perturbed ;

[0097] Normalization processing unit: used to normalize the enhanced input features:

[0098] F normalized =LayerNorm(F augmented );

[0099] The training modules include:

[0100] Comprehensive feature matrix forming unit: used to integrate comprehensive features and convert the Mamba preprocessed features F mamba And the feature F after data enhancement normalized Splice to form a comprehensive feature matrix:

[0101] F combined =[F mamba , F normalized ];

[0102] Mask matrix construction unit: used to construct the mask matrix M, whose element m ij Indicates whether the jth position of the i-th sequence sample is a filling part. The part to be filled refers to the sample sequence that has no click behavior at the current position:

[0103]

[0104] The mask technology application unit is used to apply the mask technology. During the training process, the mask matrix M is applied to train only the features of the non-filled part:

[0105] F masked =F combined ⊙M

[0106] Among them, ⊙ represents the element-by-element product;

[0107] The optimization modules include:

[0108] Selection unit: used to select appropriate main loss function and auxiliary loss function and

[0109]

[0110] in, Indicates that based on F combined The predicted value of Only indicates that based on F normalized The predicted value of

[0111] Training unit: used to train the method using stochastic gradient descent and Adam optimizer;

[0112] Optimization and evaluation unit: After the training is completed, the validation set is used to further optimize the internal parameters of the Mamba module and the relevant feature representation parameters, and finally the test set is used to evaluate the quality of the method.

[0113] The advantages of the present invention are: by using Mamba to dynamically adjust parameters of the representation vector of the user's click behavior sequence, selectively propagate or forget information, and improve the efficiency of processing long sequence information without reducing the prediction accuracy. In addition, relying on data enhancement technology, the user's click behavior sequence is processed in a diversified manner to further capture the user's potential diversified interests and achieve efficient prediction of the user's click behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] Figure 1 It is a flowchart of a method for predicting a user click behavior sequence based on Mamba and data enhancement according to an embodiment of the present invention. DETAILED DESCRIPTION

[0115] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0116] See also Figure 1 The user click behavior sequence prediction method based on Mamba and data enhancement of the present invention comprises the following steps:

[0117] Step S1: Collect user, item, item category, user click behavior sequence information, perform data cleaning, format the data into a unified form and divide the corresponding fields. Secondly, split the data set into training set, validation set and test set, and initialize the representation vectors of users, items, categories and candidate set items.

[0118] Step S2: Define and design a Mamba module suitable for the user click behavior sequence, use the selective propagation and forgetting mechanism of the Mamba module to perform dynamic parameter adjustment, and preprocess the user click behavior sequence and extract behavior features.

[0119] Step S3: Define and design data enhancement methods, generate diverse data samples, and increase the amount of training data. This step aims to improve the generalization ability of prediction results, and is used to mine the diversity of users' potential interests by formatting click behavior data into a single type.

[0120] Step S4: Integrate the results of Mamba preprocessing and data augmentation, and use masking technology in the training process to only train partial data of the non-filled part that can represent the click behavior in the corresponding batch to achieve effective training.

[0121] Step S5: Calculate the error using a suitable loss function and optimize the parameters based on the loss function. Evaluate the performance of the method on the validation set and adjust the parameters to optimize the performance of the method. Finally, evaluate the performance of the method using the test set.

[0122] Step S1 specifically includes:

[0123] Step 101: Collect the user's click behavior records on different e-commerce platforms. Including user ID set U = {u1, u2, ..., u |U|}、Product ID set G={g1,g2,...,g |G|} and product category ID set C = {c1, c2, ..., c |C|}, construct a mapping relationship from product ID to category ID. Collect the list of products or product categories that each user has clicked on S = {g1, g2, ..., g k}, forming the user's historical click sequence. Collect user context information, including time, location, device used, etc., to form a context set V = {v1, v2, ..., v |V|}.

[0124] Step 102: Organize the data into a unified format, including at least the following fields: prediction label (indicating whether the user clicks on a certain product), user ID (identifying the uniqueness of the user), product ID (the product currently clicked or in the prediction candidate set), product category ID (the product category currently clicked or in the prediction candidate set), a list of historical product IDs clicked by the user or a list of historical product category IDs, and a context classification field.

[0125] Step 103: Split the data set into a training set, a validation set, and a test set in a ratio of 7:1:2. The training set selects the first k items and their categories from the user's historical click sequence as input features, and the last item and its category as a label. The validation set is used for intermediate evaluation and parameter adjustment. The test set retains the last click behavior of each user for testing, forming a candidate item set H.

[0126] Step 104: Initialize the representation vector. Initialize the user representation vector Product representation vector Category representation vector Context Representation Vector Candidate set product representation vector Where D represents the dimension of the representation vector. Next, each representation vector is stored in its own representation vector dictionary as the initial input for the subsequent part.

[0127] Step S2 includes:

[0128] Step 201: Based on the design concept of SSM in the Mamba module, define a state space model (UBSSM) suitable for user behavior sequences. Define the original user history click behavior sequence at time t as The click behavior sequence combined with context information for semantic enhancement is: By hiding the state Mapped into the final representation vector sequence Where N is the length of the original behavior sequence:

[0129] h′(t)=A·h(t)+B·x(t)+α·C·x aug (t)

[0130] y(t)=E·h(t)

[0131] Here and is a learnable parameter matrix, and α is a predefined parameter used to control the degree of semantic enhancement. In order to process discrete user behavior sequences rather than continuous functions, it is necessary to discretize them using a step size Δ. The discretized UBSSM expression is as follows:

[0132]

[0133] y k =E·h k

[0134] in,

[0135]

[0136]

[0137] By converting the parameters from continuous form (Δ, A, B, C, E) to discrete form The calculation is performed in a linear recursive manner to improve the calculation efficiency.

[0138] Step 202: Based on the design concept of the feedforward network in the Mamba module, define and design a User Behavior-wise Feed-Forward Network (UBFFN) to further capture the user's potential interests through the hidden layer H:

[0139] UBFFN(H)=Dice(H·W (1)+b (1) )·W (2) +b (2)

[0140] in, Here, the Dice activation function is used to adaptively adjust the correction point according to the distribution of the input data, so that its value is set to the mean of the input, which can better handle the problem of different input data distributions in each layer. At the same time, Dice's smooth switching mechanism helps capture more subtle user click behavior characteristics.

[0141] Step 203: Define and design a new Mamba module. Combine UBSSM and UBFFN to build a new Mamba module to better capture the complex relationships and potential patterns in the user's click behavior sequence.

[0142] Step 204: Configure the Mamba module and embed it into an appropriate network structure, including a single-layer and multi-layer stacking structure, to meet the modeling requirements from simple behavior sequences to complex behavior sequences, and retain the Mamba hardware-aware parallel algorithm to achieve efficient computing. For a relatively simple user behavior sequence, a single-layer Mamba module can be used to embed the combined model of SSM and UBFFN into one layer. For a complex user behavior sequence, a multi-layer stacking structure can be used, that is, a stack of multiple Mamba modules, and the output of each module is used as the input of the next module, and finally the Mamba pre-processed feature representation F is obtained. mamba .

[0143] Step 205: Add residual connections between each layer of Mamba modules, that is, directly add the input to the output. This connection method can alleviate the gradient vanishing problem and improve the training efficiency of deep networks. At the same time, the output of each layer is layer normalized, that is, the output of each layer is normalized so that the output of each layer has the same mean and variance. This normalization strategy can stabilize the training process and improve the convergence speed and effect.

[0144] Step S3 includes:

[0145] Step 301: Given the target item feature representation T and the feature representation H corresponding to the historical click behavior sequence, n}, construct the basic enhanced behavior feature representation:

[0146] F base =[T, H, TH, T⊙H]

[0147] where ⊙ represents the element-wise product.

[0148] Step 302: Generate Gaussian noise N~N(0, σ 2 ), random noise is added to the historical click sequence to form a perturbed sequence representation:

[0149] H perturbed =H+N

[0150] Step 303: Combine the target item feature representation T and the perturbed sequence representation H perturbed Perform concatenation to generate perturbation feature representation:

[0151] F perturbed =[T,H perturbed , TH perturbed , T⊙H perturbed ]

[0152] Step 304: Merge the base enhancement features with the perturbation features to form the final enhanced input feature representation:

[0153] F augmented =F base +F perturbed

[0154] Step 305: Normalize the enhanced input features to make their mean and variance consistent, so that more stable and deeper feature extraction can be performed in subsequent network modules:

[0155] F normalized =LayerNorm(F augmented )

[0156] Step S4 includes:

[0157] Step 401: Integrate comprehensive features. mamba And the feature F after data enhancement normalized Splice to form a comprehensive feature matrix:

[0158] F combined =[F mamba , F normalized ]

[0159] Step 402: Construct a mask matrix M. Its element m ij Indicates whether the jth position of the i-th sequence sample is a filling part (the part to be filled refers to the sample sequence that has no click behavior at the current position):

[0160]

[0161] Step 403: Apply masking technology. During the training process, the mask matrix M is applied to train only the features of the non-filled part:

[0162] F masked =F combined ⊙M

[0163] where ⊙ represents the element-wise product.

[0164] Step S5 includes:

[0165] Step 501: Select appropriate main loss function and auxiliary loss function and

[0166]

[0167] in, Indicates that based on F combined The predicted value of Only indicates that based on F normalized The predicted value of .

[0168] Step 502: Use the stochastic gradient descent method and the Adam optimizer to train the method.

[0169] Step 503: After the training is completed, the validation set is used to further optimize the internal parameters of the Mamba module and the relevant feature representation parameters, and finally the test set is used to evaluate the quality of the method.

[0170] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A user click behavior sequence prediction method based on Mamba and data enhancement, characterized by: The steps include: Step S1: Collect information about users, items, item categories, and user click behavior sequences, perform data cleaning, format the data into a unified form, and divide the corresponding fields. Secondly, split the data set into training set, validation set, and test set, and initialize the representation vectors of users, items, categories, and candidate set items. Step S2: define and design a Mamba module suitable for the user click behavior sequence, use the selective propagation and forgetting mechanism of the Mamba module to perform dynamic parameter adjustment, preprocess the user click behavior sequence and extract behavior features, specifically including: Step 201: Based on the design concept of SSM in the Mamba module, define a state space model suitable for user behavior sequences, and define the original user history click behavior sequence at time t as The click behavior sequence combined with context information for semantic enhancement is: , through the hidden state Mapped into the final representation vector sequence Where N represents the length of the original behavior sequence: h ′ (t)=A·h(t)+B·x(t)+α·C·x aug (t) y(t)=E·h(t) Here and is a learnable parameter matrix, α is a predefined parameter used to control the degree of semantic enhancement. In order to process discrete user behavior sequences rather than continuous functions, it needs to be discretized by step size Δ. The discretized UBSSM expression is as follows: y k =E·h k in, By converting the parameters from continuous form (Δ, A, B, C, E) to discrete form The calculation is performed in a linear recursive manner; Step 202: Based on the design concept of the feedforward network in the Mamba module, define and design a behavior-aware feedforward network to further capture the user's potential interests through the hidden layer H: UBFFN(H)=Dice(H·W (1) +b (1) )·W (2) +b (2) in, Use the Dice activation function to adaptively adjust the correction point according to the distribution of the input data, so that its value is set to the mean of the input; Step 203: define and design a new Mamba module, combine UBSSM and UBFFN, and build a new Mamba module; Step 204: Configure the Mamba module and embed it into an appropriate network structure, including a single-layer and multi-layer stacked structure, and retain the parallel algorithm of Mamba hardware awareness. The output of each module is used as the input of the next module, and finally the feature representation F preprocessed by Mamba is obtained. mamba ; Step 205: Add residual connections between each layer of Mamba modules, and perform layer normalization on the output of each layer; Step S3: define and design data enhancement methods to generate diverse data samples and increase the amount of training data; Step S4: Integrate the results of Mamba preprocessing and data augmentation, and use masking technology in the training process to train only the non-filled part of the data that can represent the click behavior in the corresponding batch; Step S5: Calculate the error using a suitable loss function, optimize the parameters based on the loss function, evaluate the performance of the method on the validation set, adjust the parameters to optimize the performance of the method, and finally, evaluate the performance of the method using the test set.

2. The method for predicting user click behavior sequence based on Mamba and data enhancement as claimed in claim 1, characterized in that: Step S1 specifically includes: Step 101: Collect the user's click behavior records on different e-commerce platforms, including the user ID set U = {u1,u2,…,u |U| }、Product ID set G={g1,g2,…,g |G| } and product category ID set C = {c1,c2,…,c |C| }, construct a mapping relationship from product ID to category ID, and collect the list of products or product categories clicked by each user S = {g1, g2, ..., g k }, forming the user's historical click sequence, collecting user context information, including time, location, and device information, and forming a context set V = {v1, v2, ..., v |V| }; Step 102: Arrange the data into a unified format, including at least the following fields: a prediction label indicating whether the user clicks on a certain product, a unique user ID that identifies the user, the product ID of the currently clicked product or the prediction candidate set, the product category ID of the currently clicked product or the prediction candidate set, a list of historical product IDs clicked by the user or a list of historical product category IDs, and a context classification field; Step 103: Split the data set into a training set, a validation set, and a test set. The training set is to select the first k products and their categories from the user's historical click sequence as input features, and the last product and its category as a label. The validation set is used for intermediate evaluation and parameter adjustment. The test set retains the last click behavior of each user for testing, forming a candidate product set H. Step 104: Initialize the representation vector and the user representation vector Product representation vector Category representation vector Context Representation Vector Candidate set product representation vector Where D represents the dimension of the representation vector. Then, each representation vector is stored in its own representation vector dictionary as the initial input for the subsequent part.

3. The method for predicting user click behavior sequence based on Mamba and data enhancement as claimed in claim 1, characterized in that: In step 204, for a relatively simple user behavior sequence, a single-layer Mamba module is used to embed the combined model of SSM and UBFFN into one layer, and for a complex user behavior sequence, a multi-layer stacking structure is used.

4. The method for predicting user click behavior sequence based on Mamba and data enhancement as claimed in claim 1, characterized in that: Step S3 include: Step 301: Given the target item feature representation T and the feature representation H corresponding to the historical click behavior sequence, n }, construct the basic enhanced behavior feature representation: F base =[T,H,T-H,T⊙H] Among them, ⊙ represents the element-by-element product; Step 302: Generate Gaussian noise N~N(0,σ 2 ), add random noise to the historical click sequence to form a perturbed sequence representation: H perturbed =H+N; Step 303: Combine the target item feature representation T and the perturbed sequence representation H perturbed Perform concatenation to generate perturbation feature representation: F perturbed =[T,H perturbed ,T-H perturbed ,T⊙H perturbed ]; Step 304: Merge the base enhancement features with the perturbation features to form the final enhanced input feature representation: F augmented =F base +F perturbed ; Step 305: Normalize the enhanced input features: F normalized =LayerNorm(F augmented )。 5. The method for predicting user click behavior sequence based on Mamba and data enhancement as claimed in claim 4, characterized in that: Step S4 include: Step 401: Integrate comprehensive features and transform the Mamba pre-processed features F mamba And the feature F after data enhancement normalized Splice to form a comprehensive feature matrix: F combined =[F mamba ,F normalized ]; Step 402: Construct a mask matrix M, whose element m ij Indicates whether the jth position of the i-th sequence sample is a filling part. The part to be filled refers to the sample sequence that has no click behavior at the current position: Step 403: Apply masking technology. During the training process, apply the mask matrix M to train only the features of the non-filled part: F masked =F combined ⊙M where ⊙ represents the element-wise product.

6. The method for predicting user click behavior sequence based on Mamba and data enhancement as claimed in claim 5, characterized in that: Step S5 includes: Step 501: Select appropriate main loss function and auxiliary loss function and in, Indicates that based on F combined The predicted value of Only indicates that based on F normalized The predicted value of Step 502: training the method using stochastic gradient descent and Adam optimizer; Step 503: After the training is completed, the validation set is used to further optimize the internal parameters of the Mamba module and the relevant feature representation parameters, and finally the test set is used to evaluate the quality of the method.

7. A user click behavior sequence prediction system based on Mamba and data enhancement, characterized by: include: Information collection module: used to collect information about users, items, item categories, and user click behavior sequences, perform data cleaning, format the data into a unified form, and divide the data into corresponding fields. Secondly, the data set is split into training set, validation set, and test set, and the representation vectors of users, items, categories, and candidate set items are initialized. Mamba module design module: used to define and design Mamba modules suitable for user click behavior sequences, use the selective propagation and forgetting mechanism of Mamba modules to adjust parameters dynamically, pre-process user click behavior sequences and extract behavior features, including: State space model definition unit: used to define the state space model applicable to the user behavior sequence according to the design concept of SSM in the Mamba module, and define the original user history click behavior sequence at time t as The click behavior sequence combined with context information for semantic enhancement is: By hiding the state Mapped into the final representation vector sequence Where N represents the length of the original behavior sequence: h ′ (t)=A·h(t)+B·x(t)+α·C·x aug (t) y(t)=E·h(t) Here and is a learnable parameter matrix, α is a predefined parameter used to control the degree of semantic enhancement. In order to process discrete user behavior sequences rather than continuous functions, it needs to be discretized by step size Δ. The discretized UBSSM expression is as follows: y k =E·h k in, By converting the parameters from continuous form (Δ, A, B, C, E) to discrete form The calculation is performed in a linear recursive manner; Potential interest capture unit: It is used to define and design a behavior-aware feedforward network based on the design concept of the feedforward network in the Mamba module, and further capture the user's potential interest through the hidden layer H: UBFFN(H)=Dice(H·W (1) +b (1) )·W (2) +b (2) in, Use the Dice activation function to adaptively adjust the correction point according to the distribution of the input data, so that its value is set to the mean of the input; New module building unit: used to define and design new Mamba modules, combining UBSSM and UBFFN to build new Mamba modules; Configuration unit: used to configure the Mamba module and embed it into the appropriate network structure, including single-layer and multi-layer stacking structures, and retain the parallel algorithm of Mamba hardware awareness. The output of each module is used as the input of the next module, and finally the feature representation F preprocessed by Mamba is obtained. mamba ; Connection unit: used to add residual connections between each layer of Mamba modules and perform layer normalization on the output of each layer; Data sample generation module: define and design data enhancement methods, generate diverse data samples, and increase the amount of training data; The training module is used to integrate the results of Mamba preprocessing and data enhancement. During the training process, masking technology is used to train only the non-filled part of the data that can represent the click behavior in the corresponding batch; The optimization module is used to calculate the error using a suitable loss function, optimize the parameters according to the loss function, evaluate the system performance on the validation set, adjust the parameters to optimize the system performance, and finally, evaluate the system performance using the test set.

8. The user click behavior sequence prediction system based on Mamba and data enhancement as claimed in claim 7, characterized in that: The information collection module specifically includes: The click behavior recording unit is used to collect the click behavior records of users on different e-commerce platforms, including the user ID set U = {u1,u2,…,u |U| }、Product ID set G={g1,g2,…,g |G| } and product category ID set C = {c1,c2,…,c |C| }, construct a mapping relationship from product ID to category ID, and collect the list of products or product categories clicked by each user S = {g1, g2, ..., g k }, forming the user's historical click sequence, collecting user context information, including time, location, and device information, and forming a context set V = {v1, v2, ..., v |V| }; A formatting unit is used to organize the data into a unified format, which at least includes the following fields: a prediction label indicating whether a user clicks on a certain product, a unique user ID that identifies the user, a product ID currently being clicked or in a prediction candidate set, a product category ID currently being clicked or in a prediction candidate set, a list of historical product IDs clicked by the user or a list of historical product category IDs, and a context classification field; The splitting unit is used to split the data set into a training set, a validation set, and a test set. The training set selects the first k products and their categories from the user's historical click sequence as input features, and the last product and its category as a label. The validation set is used for intermediate evaluation and parameter adjustment. The test set retains the last click behavior of each user for testing to form a candidate product set H. Vector representation unit, used to initialize the representation vector, initialize the user representation vector Product representation vector Category representation vector Context Representation Vector Candidate set product representation vector Where D represents the dimension of the representation vector. Then, each representation vector is stored in its own representation vector dictionary as the initial input for the subsequent part.

9. The user click behavior sequence prediction system based on Mamba and data enhancement as claimed in claim 7, characterized in that: The data sample generation module includes: Feature representation unit: used to represent the feature representation T of the target item and the feature representation H corresponding to the historical click behavior sequence = {h1,h2,…,h n }, construct the basic enhanced behavior feature representation: F base =[T,H,T-H,T⊙H] Among them, ⊙ represents the element-by-element product; Sequence generation unit: used to generate Gaussian noise N~N(0,σ 2 ), add random noise to the historical click sequence to form a perturbed sequence representation: H perturbed =H+N; Concatenation unit: used to combine the target item feature representation T with the perturbed sequence representation H perturbed Perform concatenation to generate perturbation feature representation: F perturbed =[T,H perturbed ,T-H perturbed ,T⊙H perturbed ]; Merging unit: used to merge the basic enhanced features with the perturbation features to form the final enhanced input feature representation: F augmented =F base +F perturbed ; Normalization processing unit: used to normalize the enhanced input features: F normalized =LayerNorm(F augmented )。 10. The user click behavior sequence prediction system based on Mamba and data enhancement as claimed in claim 9, characterized in that: The training modules include: Comprehensive feature matrix forming unit: used to integrate comprehensive features, and convert the Mamba pre-processed features F mamba And the feature F after data enhancement normalized Splice to form a comprehensive feature matrix: F combined =[F mamba ,F normalized ]; Mask matrix construction unit: used to construct the mask matrix M, whose element m ij Indicates whether the jth position of the i-th sequence sample is a filling part. The part to be filled refers to the sample sequence that has no click behavior at the current position: The mask technology application unit is used to apply the mask technology. During the training process, the mask matrix M is applied to train only the features of the non-filled part: F masked =F combined ⊙M Among them, ⊙ represents the element-by-element product; The optimization modules include: Selection unit: used to select appropriate main loss function and auxiliary loss function and in, Indicates that based on F combined The predicted value of Only indicates that based on F normalized The predicted value of Training unit: used to train the system using stochastic gradient descent and Adam optimizer; Optimization and evaluation unit: After training is completed, the validation set is used to further optimize the internal parameters of the Mamba module and related feature representation parameters, and finally the test set is used to evaluate the quality of the system.

Images