A feature-based data enhancement method for sequence recommendation

Abstract

The present invention belongs to the technical field of sequence recommendation, and in particular relates to a feature-based sequence recommendation data enhancement method, comprising the following steps: pre-processing the obtained original user sequence, mapping all items of the original user sequence to a high-dimensional space; clustering all pre-processed items using the K-means clustering algorithm to obtain the embedding distribution of all items; obtaining the spatial distribution loss by calculating the distance between the target item in the user sequence and the cluster center after clustering; back-propagating the spatial distribution loss to obtain the gradient value of each dimension of embedding; and using the gradient value of each dimension of embedding to perform data enhancement based on the dropout method. This method analogizes the feature selection process of the item to the item embedding space, calculates the importance ranking of each dimension of the item embedding, and uses this importance to perform an embedding-level data enhancement process. The present invention retains the semantics and structure of the sequence through a vector-level enhancement method, and can generate enhanced data of higher quality.

Description

Technical Field

[0001] The present invention belongs to the technical field of sequence recommendation, and in particular relates to a feature-based sequence recommendation data enhancement method. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] For sequential recommendation tasks, the basic approach is to leverage historical user interaction sequences to derive high-quality user representations, which are then used to recommend items of interest to the user. However, these approaches typically optimize the large number of model parameters using the item prediction task, which is susceptible to data sparsity. When the dataset is relatively sparse, these methods may fail to generate accurate user representations. Data augmentation methods can alleviate this problem.

[0004] Data augmentation methods have achieved excellent results in image classification, natural language processing, speech recognition and other fields. In the recommendation field, inspired by self-supervised tasks, CL4SRec designed a simple data augmentation method to enhance the original data by cropping, masking or reordering the sequence data. In terms of graph structure, NCL enhances the graph collaborative filtering method by introducing semantic neighbors to reduce the impact of data sparsity. SGL uses randomly sampled nodes and edges to enhance the interaction graph to form contrast pairs. At present, whether it is a sequential recommendation method or a graph-based method, most data augmentation methods directly process the original sequence and generate related augmented data for training.

[0005] In the field of sequential recommendation, existing data augmentation methods mostly process the original data, obtaining relevant data through methods such as cropping, masking, or reordering. However, this approach may result in semantic inconsistencies between the new sequence and the original sequence. For example, due to the random nature of data cropping or masking, important items in the original sequence may be discarded, which can significantly interfere with user modeling. Therefore, if the augmentation level is not properly controlled, this augmentation method, which processes the original data, will inevitably lead to information loss and introduce noise and ambiguity, thus affecting recommendation performance. Furthermore, previous data augmentation methods only utilize their own augmented data for comparative training, ignoring potential connections between users. There are often similar users in a user group. If only the augmented data of the user is used for comparative training, other users with similar interests will be classified as negative samples, which will lead to reduced recommendation performance. Summary of the Invention

[0006] To overcome the above-mentioned deficiencies in the prior art, the present invention provides a feature-based sequence recommendation data enhancement method.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] A first aspect of the present invention provides a feature-based sequence recommendation data enhancement method, comprising the following steps:

[0009] Preprocess the obtained original user sequence and map all items of the original user sequence into a high-dimensional space;

[0010] All preprocessed items are clustered using the K-means clustering algorithm to obtain the embedding distribution of all items;

[0011] The spatial distribution loss is obtained by calculating the distance between the target item in the user sequence and the cluster center after clustering;

[0012] Backpropagate the spatial distribution loss to obtain the gradient value of each dimension of the embedding;

[0013] The gradient value of each dimension is embedded and data enhancement is performed based on the dropout method.

[0014] Furthermore, the preprocessing also includes: introducing a position encoding matrix to add the item embedding and the position encoding vector as model input.

[0015] Furthermore, the method further includes performing a sequence recommendation task based on the enhanced user sequence, comprising the following steps:

[0016] Input the enhanced user sequence into the encoder to obtain the user representation;

[0017] Leveraging user representations for recommendation prediction;

[0018] or,

[0019] The method also includes a contrast training task, comprising the following steps:

[0020] Perform data augmentation on the acquired dataset;

[0021] Based on the spatial distribution sampling strategy, the clustering results of the data set are divided to obtain semantically similar sequences of user sequences;

[0022] The enhanced semantically similar sequence is input into the encoder to obtain similar user representations;

[0023] Similar user representations and user representations are used together as positive sample pairs for comparative learning. All other enhanced samples in the same training batch are input into the encoder as negative samples. Comparative training is performed using the user's positive and negative samples to obtain more accurate user representations.

[0024] Furthermore, the sampling strategy based on spatial distribution includes: selecting the last item in the user sequence as the target item, and classifying the user sequences in the same cluster as the target item as semantically similar user sequences;

[0025] If two users have the same target project, the one with the same priority will be selected.

[0026] Furthermore, the comparative training includes: using dot product to measure the similarity between each vector, and for the data of the training batch B, training is performed using the following formula:

[0027]

[0028] Among them, S - represents the negative sample set, h′ i ,h′ i,s They represent the enhanced user representation and the representation of users with similar interests, μ is a hyperparameter, and T represents vector transpose;

[0029] Furthermore, the spatial distribution loss is:

[0030]

[0031] Among them, τ represents the same derivative of each Gaussian distribution, v represents the predicted item, and c i It is the vector representation of the cluster center after clustering, and K represents the number of clusters;

[0032] For a given item and its spatial distribution loss, the spatial distribution loss is back-propagated without updating the gradient and parameters. The absolute value of the spatial distribution loss to the item gradient is used as the basis for importance judgment.

[0033] Furthermore, the method of using user representation to make recommendation predictions includes:

[0034] The inner product is used to calculate the predicted score of the interaction between the user representation and each item in the item set.

[0035] The cross entropy loss function is used to calculate the error of the prediction task:

[0036]

[0037] Among them, b is the index of the training sample, B is the number of sequences in a batch, and y n represents the one-hot encoding of the actual next item, represents the prediction score;

[0038] Alternatively, the method further includes a multi-task training strategy, comprising the following steps:

[0039] Joint training comparison training tasks, data enhancement tasks and sequence recommendation prediction tasks,

[0040] Adopt multi-task training strategy to optimize the objective function;

[0041] The objective function is as follows:

[0042] L=L Rec +λ1L s +λ2L scl ,

[0043] Among them, λ1 and λ2 are hyperparameters that control the strength of the two proposed tasks, L s is the spatial distribution loss function; L Rec Represents the error of the prediction task; L scl represents the error of contrastive training.

[0044] Alternatively, the dropout-based enhancement method includes: embedding h 0 Perform dropout operation to obtain to I v (x) is sorted, and the regions whose importance ranking is higher than the given ratio threshold are retained. Random dropout is performed on the remaining regions. The enhancement results are as follows:

[0045]

[0046] Where M(x) represents a binary mask, and the importance score of M(x) is I v (x) The regions ranked higher than ω are assigned a value of 1, and the rest are 0;

[0047] Alternatively, the EM algorithm is used to optimize the K-means clustering model.

[0048] A second aspect of the present invention provides a feature-based sequence recommendation data enhancement system, comprising:

[0049] A preprocessing module is configured to preprocess the obtained original user sequence and map all items of the original user sequence into a high-dimensional space;

[0050] The clustering module is configured to cluster all preprocessed items using the K-means clustering algorithm to obtain the embedding distribution of all items;

[0051] The spatial distribution loss calculation module is configured to obtain the spatial distribution loss by calculating the distance between the target item in the user sequence and the cluster center after clustering;

[0052] The back-propagation module is configured to back-propagate the spatial distribution loss and derive the gradient value of each dimension of the embedding;

[0053] The data enhancement module is configured to use the gradient value of each dimension of the embedding to perform data enhancement based on the dropout method.

[0054] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the feature-based sequence recommendation data enhancement method as described in the first aspect of the present invention.

[0055] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of the feature-based sequence recommendation data enhancement method described in the first aspect of the present invention are implemented.

[0056] One or more of the above technical solutions have the following beneficial effects:

[0057] (1) The present invention provides a feature-based sequence recommendation data enhancement method, which analogizes the feature selection process of the project to the project embedding space, calculates the importance ranking of each dimension of the project embedding, and uses this importance to perform the embedding-level data enhancement process. The data enhancement method of the present invention is different from the previous method of enhancement at the data level. The idea of the present invention is to enhance at the vector level to achieve embedding-level enhancement of the data. Because the previous data-level enhancement method will destroy the original data structure, it may lose the structure and semantics of the original sequence. Therefore, the present invention retains the semantics and structure of the sequence through the vector-level enhancement method, and can generate higher-quality enhanced data.

[0058] (2) Utilizing augmented data for recommendation prediction tasks. Since contrastive learning using only augmented data can result in semantically similar data being classified as negative samples, the present invention provides a sampling process based on spatial distribution. This involves performing contrastive training on semantically similar data and augmented data. By contrastive training on augmented data, a more accurate user representation is formed, enhancing the robustness of the model. Finally, by jointly training the three tasks, recommendation performance is improved.

[0059] (3) To address the problem that existing data augmentation methods may introduce noisy augmented data, which affects recommendation performance, this application proposes a novel contrastive learning data augmentation method. This method is based on the distribution of items in the embedding space. It performs data augmentation by finding the most important feature dimensions that determine the spatial distribution of the items, retaining these important dimensions, and randomly discarding unimportant information. At the same time, we consider the semantic relevance between similar users and propose a new sampling strategy. To address the data sparsity problem, we propose an embedding-level solution, which effectively alleviates the data sparsity problem and improves recommendation performance.

[0060] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0062] Figure 1 This is a flow chart of the method of the first embodiment. DETAILED DESCRIPTION

[0063] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0064] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0065] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0066] Example 1

[0067] This embodiment discloses a data augmentation method for sequential recommendation. Structurally, the method comprises three training tasks: a recommendation prediction task, a distribution loss task for embedding items in a high-dimensional space, and a data augmentation task. These three tasks complement each other and work together.

[0068] For the recommendation prediction task, the present invention generates a new vector representation by performing data enhancement on the original user sequence, then generates a user representation through a Transformer-based encoder, and then uses the user representation to perform recommendation prediction.

[0069] The core idea of data augmentation in this paper is to retain the valid parts of the data while discarding the dispensable parts, which can improve the robustness of the augmentation method. For the important parts of the data, the feature distribution of the items is analogized to the spatial distribution. The spatial distribution loss is used to quantify the importance of the embedding dimension, and this importance is used to control the degree of data augmentation.

[0070] Feature selection is a crucial process in recommendation systems, yet existing methods haven't paid much attention to this aspect. Typically, they simply feed all features into a deep model architecture, allowing the model to automatically extract features. This approach doesn't discard useless features, which could interfere with model reasoning. Therefore, extracting key features that represent items is a crucial issue in sequential recommendation.

[0071] In the current modeling methods based on deep learning, they are all based on embedded neural networks, in which each item is mapped into a dense vector in a high-dimensional space. These vectors represent sparse features with high cardinality, and the embedded vectors can express certain features of the corresponding objects, such as item categories, attributes, etc. Therefore, the distribution of vectors in high-dimensional space also reflects the feature distribution of the items, and items with similar features will have closer vector representations in high-dimensional space. The spatial distribution characteristics of an item embedding are determined by several of the most important features. For example, the brand, price, material, etc. of a piece of clothing are more important. If the dimensions representing these features change, it will have a greater impact on the spatial distribution of the overall vector, while other unimportant features will not have much impact on the distribution of the vector. Therefore, the present invention transforms the problem of feature selection into the problem of vector space distribution, and transforms the problem of finding key features into the problem of finding the most important dimension when the vector determines the spatial distribution.

[0072] Based on this, the present invention designs a data enhancement method that uses the spatial distribution of items to extract key features. Considering that the input data is usually mapped into a dense vector, an embedding-level data enhancement method is proposed. Specifically, it includes: obtaining the embedding distribution of all items through clustering. Since the distribution of item embeddings is determined by different item features, in each training step, the distance between the item and the cluster center after clustering is calculated to obtain the spatial distribution loss, and the loss function is back-propagated to obtain the contribution of each dimension of the item embedding in determining the spatial distribution of the item. Considering the dimension with a large gradient value, if it changes, it will have a greater impact on the final loss, while the dimension with a small gradient value, their change has little effect on the final loss. Afterwards, the gradient value of each dimension is embedded, and data enhancement is performed in an adaptive manner, so that the dimensions with high importance are always retained, and the dimensions with low importance are randomly discarded through dropout. In this way, the enhanced data obtained can maintain important features.

[0073] In addition, unlike CV and NLP tasks that treat each data instance as an independent one, users and items in recommendation tasks are essentially interrelated and dependent on each other, and users with similar interests have potential connections. If only self-supervised tasks are applied, and only the enhanced data of the same user is used as positive samples, then users with similar interests will be classified as negative samples, which will lead to a decline in model performance. Therefore, the present invention designs a new sampling strategy based on spatial distribution, which classifies users in the same cluster as the target item as positive samples to obtain semantically similar data. When performing positive sampling, a positive sample is randomly sampled from semantically similar users, and used together with the enhanced data as a positive sample pair for comparative learning. Among them, if the target items are the same, they will be given priority, and users with the same or similar target items will also have similar interests, so as to divide positive and negative samples. Finally, comparative learning tasks are performed using positive and negative sample pairs;

[0074] The specific implementation steps of the present invention are as follows:

[0075] (1) Notation and problem definition:

[0076] In the recommendation system, the most basic problem is to use the user's historical interactions to infer the user's interest preferences, so as to recommend the next item that may be interacted with. We denote the user set and item set by U and V respectively. The chronological interaction sequence of each user (u∈U) is in is the t-th interaction item of user u, |s u | is the total number of interaction items. The present invention takes the most recent T interaction items. If the length of the sequence is greater than T, the sequence will be truncated and only the most recent T interaction items will be considered. If the length is insufficient, [PAD] will be used for padding. For each user u, the purpose of the present invention is to calculate the total number of interaction items based on the given interaction sequence s. u In the case of |s u |+1 step most likely to interact with the project.

[0077] (2) Input and embedding layer:

[0078] Apply an item embedding matrix V∈R |V|×d Map all items into a high-dimensional space, where |V| represents the number of items and d is the dimension of the embedding. in is the item that user u interacted with in step t, and T is the maximum length of the sequence. As s u The embedding representation of is the embedded representation of the item that user u interacts with at time t. For simplicity, the user superscript u is omitted in the following description.

[0079] In order to maintain the relative order of the input sequence, the position encoding matrix P∈R is introduced T×d , where T represents the maximum length of all sequences. The initial input to the model is the sum of the item embedding and the position encoding vector.

[0080]

[0081] in, is the initial input vector of the model, p t is the vector representation of the position t.

[0082] (3) Comparative learning framework:

[0083] This section introduces our contrastive learning algorithm. The framework consists of two parts: a random data augmentation module and our proposed spatially distributed sampling strategy. These two parts generate two types of augmented data for user sequences for contrastive training.

[0084] (3.1) Random Data Augmentation Module

[0085] This section details the feature selection process before data enters the user encoder. We also use dropout to randomly mask unimportant features with a certain probability, retaining important features, to enhance data. The enhanced data is then sent to the user representation encoder layer for further processing to obtain the final user representation for recommendation prediction.

[0086] Formally, the prediction task objective of the present invention can be written as:

[0087]

[0088] Among them, p is the predicted probability, v represents the predicted item, and θ is a set of model parameters. As mentioned earlier, we perform feature selection at the embedding level to capture those more important feature dimensions, and achieve embedding-level data enhancement by retaining these important dimensions. The distribution of project embeddings in high-dimensional space has a certain regularity. The spatial position of each project is controlled by several main features, and the embeddings with similar main features will be closer. Therefore, starting from the spatial distribution of the project, the main features of the project can be better captured by learning the spatial neighbors of each project. Specifically, similar projects tend to fall in adjacent embedding spaces, and they can be abstractly represented by the center point of the cluster. Assuming that there are K clusters after the project is clustered, when forming the cluster variable In a recommendation system, the probability of a user interacting with an item can be rewritten as follows:

[0089]

[0090] The K-means clustering algorithm is used to obtain the cluster center representation of the project. Since this process cannot be optimized end-to-end, the EM algorithm is used to optimize the model, including: clustering all projects in the E-step to obtain K clusters. Then, in the M-step, the generated clusters are used to maximize formula (3). Assume that the project distribution follows Q(c), where Q(c i )≥0, and the parameters θ and cluster distribution Q(c) are updated through repeated iterations. Formally, the objective of the model can be rewritten as maximizing the following log-likelihood function:

[0091]

[0092] Among them, θ is a set of model parameters, v represents the predicted item, c i is the vector representation of the cluster center after clustering. We set the hyperparameter K to the number of clusters. However, the above formula is difficult to optimize. Therefore, we can construct the lower bound of formula (4) and maximize its lower bound. Then we can calculate:

[0093]

[0094] According to Jensen's inequality, the above formula:

[0095]

[0096] Where ∝ represents proportional to..., when Q(c i )=p(c i |v,θ), the inequality holds. Now, we have found the lower bound of the optimization target, but we cannot directly optimize the formula because Q(c) is unknown. Therefore, we need to use the EM algorithm. In the E-step, we use the K-means algorithm to learn the distribution of Q(c) and obtain the item prototype. It is stipulated that for item v belongs to cluster i, Q(c i |v)=1 and Q(c j |v)=0, and then perform the M-step optimization prediction task. Through alternating iterations, both Q(c) and model parameters θ will be updated.

[0097] Now we have the distribution function Q(c), in order to maximize formula (4), we also need to define p(v,c i |θ), for convenience, it is rewritten as p θ (v,c i ). Assuming that the prior distribution follows a uniform distribution and the conditional distribution given c is an isotropic Gaussian distribution with L2 normalization, then p can be θ (v,c i ) is rewritten as:

[0098]

[0099] Therefore, according to formulas (6) and (7), maximizing formula (6) is equivalent to minimizing the following spatial distribution loss function:

[0100]

[0101] Here, τ is the same derivative of each Gaussian distribution.

[0102] As can be seen from formula (8), the present invention attempts to maximize the mutual information between the project and its corresponding cluster to shorten the distance between the project and its cluster and to increase the distance between the project and other clusters. This spatial distribution of the project is closely related to its feature distribution. Each dimension of the embedding can be understood as a feature of the project. They work together to determine the spatial distribution position of the project. For this distribution, when we change some of the dimensions, some important feature dimensions will play a dominant role. Changing them may cause a large change in the overall position of the vector, while other less important areas will not cause much change. Through the loss function proposed by formula (8), by backpropagating it, according to the size of the gradient value of each dimension, the dimensions with high importance in the project embedding are obtained, and these important areas are ensured to be retained during the data enhancement process. Data enhancement is performed through this information retention strategy.

[0103] Specifically, based on formula (8), for a given item v and its spatial distribution loss L s , for L s Perform back propagation, but do not update the gradient and parameters. s The absolute value of the gradient of item v |▽ v L s ( v )| as the basis for judging importance. It can be seen that if the dimensions with high gradient values change slightly, the loss L s The impact of I is relatively large. This dimension is more important in determining the spatial distribution of projects. v (x) represents the importance score of the x-th dimension of item v.

[0104] The unsupervised contrast enhancement in the model aims to provide semantic data enhancement for each user in an unsupervised manner. In recommendation, methods such as random cropping, masking or reordering are used. Although these methods have improved the recommendation performance of the relevant models to some extent, this method may cause the semantics of the new sequence to be inconsistent with the original sequence, and in the recommendation field, the characteristics of users and items are discrete. Therefore, this data-level operation is not applicable. The enhancement process of the original data can be regarded as a process of injecting disturbances into the data center, which can improve its robustness. However, changing the original data will interfere with the original semantics. Therefore, the present invention adopts embedding-level enhancement. Embedding-level enhancement does not perform any operations on the original data. It can retain the pattern and semantic information of the original sequence, so that the generated positive sample pairs have higher confidence. Here, the present invention chooses an enhancement method based on dropout.

[0105] For the dropout-based enhancement method, it is necessary to ensure that the discarded area cannot have a large importance score. First, the embedding h 0 Perform dropout operation to obtain Afterwards I v (x) is sorted, and the present invention retains the areas whose importance ranking is higher than the given ratio threshold ω, and only performs random dropout on the remaining areas. The corresponding enhancement examples are as follows:

[0106]

[0107] Where M(x) is a binary mask, we let the importance score of M(x) be I v (x) Regions ranked higher than ω are assigned a value of 1, and the rest are assigned a value of 0.

[0108] (3.2) Sampling module

[0109] Embedding-level enhancement using dropout. In theory, by updating the input sequence twice with different dropouts, semantically similar but different enhanced samples can be generated for comparative training. However, this method will regard all enhanced samples generated by other sequences as negative samples. But the reality is that there are many users with similar interests. If they are directly classified as negative samples, it will affect the recommendation performance. Therefore, the present invention proposes a new sampling strategy. Using the previous clustering results, if the next interactive project of a user is distributed in the same cluster as another user, it is considered that their interests are similar, that is, their target projects are similar. In particular, if the target projects of two users are the same, they will be given priority.

[0110] When sampling, the sequence with the same target item will be prioritized as the positive sample of the sequence, that is, if the sequence si Target items and j Same, then s j As s i A positive sample of . If there is no i If the target item is the same as the sequence, find the target item and s from all sequences. i The target item is in the same cluster as other sequences, from which a sequence is randomly selected as s i A positive sample of .

[0111] Positive sampling: For the input sequence s i , if there is a sequence with the same target item in the dataset, we randomly select one as a semantically similar sequence s s If not, find the target item and s from all sequences. i The target item is in the same cluster as other sequences, from which a sequence is randomly selected as s i A positive sample of . And s i The enhanced representation of is another positive sample, and these two positive samples are used for comparative learning. The sequence s is given by formula (11)(12)(13) i and s s Perform user encoding to obtain the corresponding sequence representations h′ and h′ s .

[0112] Negative sampling: In order to effectively construct negative samples of enhanced sample pairs, this paper considers all other enhanced samples in the same training batch as negative samples. Assuming that the training batch size is B, after enhancement, 2|B| vectors representing {h′1,h′ 1,s ,h′2,h′ 2,s ,…,h′ |B| ,h′ |B|,s}, where the subscript is the index of the training sequence. Therefore, for each set of training positive samples, it has 2(|B|-1) negative samples. For example, for the enhanced data h′1,h′ 1,s , their corresponding negative sample sets are Similarly, if there is a sequence in the same cluster as the target item, S should be sent - Eliminate from.

[0113] (3.3) Contrastive Learning

[0114] For each user, the positive and negative samples of the user are used for training, and the dot product is used to measure the similarity between each vector. For the training batch of data B, the training is performed using the following formula:

[0115]

[0116] Among them, S - represents the negative sample set, h′ i ,h′ i,s They represent the enhanced user representation and the representation of users with similar interests, μ is a hyperparameter, and T represents vector transpose;

[0117] By contrastive training, the distance between positive samples is shortened and the distance between negative samples is distanced. This can make the target user representation closer to users with similar interests and away from users with different interests, thereby extracting more accurate user interests.

[0118] (4) User representation encoder

[0119] In sequential recommendation systems, the main idea is to describe user interests and preferences by aggregating historical interaction sequences. This paper adopts an encoder structure similar to SASRec, first converting items into embedded representations and extracting user representations through a multi-head attention mechanism. After data enhancement, the user's historical interaction sequence is sent to the user encoder layer to calculate the updated representation of each item. As the hidden representation of the encoder’s initial input sequence, the encoding process can be described as:

[0120]

[0121]

[0122]

[0123] in It is a scaling factor to prevent the product from being too large. Gelu is the activation function, W1∈R d×4d ,b1∈R 4d ,W2∈R 4d×d ,b2∈R d These are all trainable model parameters. Stack L encoder layers to get the final hidden state The last hidden state is selected as the final user representation h′.

[0124] (5) Multi-task learning

[0125] After the encoder layer, the final user representation h′ can be obtained. The inner product is used to calculate the probability of the user interacting with each item. The prediction score is calculated as follows:

[0126]

[0127] in, We then use the cross entropy loss function to calculate the error of the prediction task:

[0128]

[0129] Among them, b is the index of the training sample, B is the number of sequences in a batch, and y n Represents the one-hot encoding of the true next item.

[0130] A multi-task training strategy is used to optimize the objective function, and the contrastive learning task is jointly trained using formula (16). Formally, the objective function is as follows:

[0131] L=L Rec +λ1L s +λ2L scl (16)

[0132] Among them, λ1 and λ2 are hyperparameters that control the strength of the two proposed tasks;

[0133] The three tasks interact with each other to better optimize the final objective function

[0134] Example 2

[0135] This embodiment discloses a feature-based sequence recommendation data enhancement system, comprising: a preprocessing module configured to preprocess the acquired original user sequence and map all items in the original user sequence into a high-dimensional space;

[0136] The clustering module is configured to cluster all preprocessed items using the K-means clustering algorithm to obtain the embedding distribution of all items;

[0137] The spatial distribution loss calculation module is configured to obtain the spatial distribution loss by calculating the distance between the target item in the user sequence and the cluster center after clustering;

[0138] The back-propagation module is configured to back-propagate the spatial distribution loss and derive the gradient value of each dimension of the embedding;

[0139] The data enhancement module is configured to use the gradient value of each dimension of the embedding to perform data enhancement based on the dropout method.

[0140] Example 3

[0141] The purpose of this embodiment is to provide a computer-readable storage medium.

[0142] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a feature-based sequence recommendation data enhancement method as described in the first embodiment of the present disclosure.

[0143] Example 4

[0144] The purpose of this embodiment is to provide an electronic device.

[0145] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of a feature-based sequence recommendation data enhancement method as described in the first embodiment of the present disclosure are implemented.

[0146] The steps involved in the apparatuses of Examples 2, 3, and 4 above correspond to those of Method Example 1. For detailed implementation, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and causing the processor to perform any of the methods of the present invention.

[0147] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0148] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A feature-based sequence recommendation data enhancement method, characterized in that: The following steps are involved: Preprocess the obtained original user sequence and map all items of the original user sequence into a high-dimensional space; All preprocessed items are clustered using the K-means clustering algorithm to obtain the embedding distribution of all items; By calculating the distance between the items in the user sequence and the cluster center after clustering, the spatial distribution loss is obtained. The spatial distribution loss is: Among them, τ represents the same derivative of each Gaussian distribution, v represents the predicted item, and c i It is the vector representation of the cluster center after clustering, and K represents the number of clusters; For a given item and its spatial distribution loss, backpropagate the spatial distribution loss without updating the gradient and parameters. The absolute value of the spatial distribution loss to the item gradient is used as the basis for judging importance. Backpropagate the spatial distribution loss to obtain the gradient value of each dimension of the embedding; Using the gradient value embedded in each dimension, data enhancement is performed based on a dropout method. The method includes performing a sequence recommendation task based on the enhanced user sequence, including the following steps: Input the enhanced user sequence into the encoder to obtain the user representation; Leveraging user representations for recommendation prediction.

2. The feature-based sequence recommendation data enhancement method according to claim 1, characterized in that: The preprocessing also includes: introducing a position encoding matrix to add the item embedding and the position encoding vector as model input.

3. The feature-based sequence recommendation data enhancement method according to claim 1, wherein: The method includes a contrast training task, comprising the following steps: Perform data augmentation on the acquired dataset; Based on the spatial distribution sampling strategy, the clustering results of the data set are divided to obtain semantically similar sequences of user sequences; Semantically similar sequences and user sequences are used together as positive sample pairs for contrastive learning, and all other enhanced samples in the same training batch are used as negative samples. Comparative training is performed using the user's positive and negative samples to obtain more accurate user representation.

4. The feature-based sequence recommendation data enhancement method according to claim 3, wherein: The sampling strategy based on spatial distribution includes: selecting the last item in the user sequence as the target item, and classifying the user sequences in the same cluster as the target item as semantically similar user sequences; If two users have the same target project, the one with the same priority will be selected.

5. The feature-based sequence recommendation data enhancement method according to claim 3, wherein: The comparative training includes: using dot product to measure the similarity between each vector, and for data with a training batch of B, training is performed using the following formula: Among them, S - represents the negative sample set, h′ i ,h′ i,s They represent the enhanced user representation and the representation of users with similar interests respectively, μ is a hyperparameter, and T represents vector transpose.

6. The feature-based sequence recommendation data enhancement method according to claim 1, characterized in that: The method of using user representation to perform recommendation prediction includes: The inner product is used to calculate the predicted score of the interaction between the user representation and each item in the item set. The cross entropy loss function is used to calculate the error of the prediction task: Among them, b is the index of the training sample, B is the number of sequences in a batch, and y n represents the one-hot encoding of the actual next item, represents the prediction score; The method also includes a multi-task training strategy, comprising the following steps: Joint training comparison training tasks, data enhancement tasks and sequence recommendation prediction tasks, Adopt multi-task training strategy to optimize the objective function; The objective function is as follows: L=L Rec +λ1L s +λ2L scl , Among them, λ1 and λ2 are hyperparameters that control the strength of the two proposed tasks, L s is the spatial distribution loss function; L Rec Represents the error of the prediction task; L scl represents the error of contrast training; The dropout-based enhancement method includes: embedding h 0 Perform dropout operation to obtain to I v (x) is sorted, and the regions whose importance ranking is higher than the given ratio threshold are retained. Random dropout is performed on the remaining regions. The enhancement results are as follows: Where M(x) represents a binary mask, and the importance score of M(x) is I v (x) The regions ranked higher than ω are assigned a value of 1, and the rest are 0; The EM algorithm is used to optimize the K-means clustering model.

7. A feature-based sequence recommendation data enhancement system, characterized in that: include: A preprocessing module is configured to preprocess the obtained original user sequence and map all items of the original user sequence into a high-dimensional space; The clustering module is configured to cluster all preprocessed items using the K-means clustering algorithm to obtain the embedding distribution of all items; The spatial distribution loss calculation module is configured to calculate the distance between the target item in the user sequence and the cluster center after clustering to obtain the spatial distribution loss. The spatial distribution loss is: Among them, τ represents the same derivative of each Gaussian distribution, v represents the predicted item, and c i It is the vector representation of the cluster center after clustering, and K represents the number of clusters; For a given item and its spatial distribution loss, backpropagate the spatial distribution loss without updating the gradient and parameters. The absolute value of the spatial distribution loss to the item gradient is used as the basis for judging importance. The back-propagation module is configured to back-propagate the spatial distribution loss and derive the gradient value of each dimension of the embedding; The data enhancement module is configured to utilize the gradient value of each dimension embedded in the data to perform data enhancement based on the dropout method, wherein the method includes performing a sequence recommendation task based on the enhanced user sequence, including the following steps: Input the enhanced user sequence into the encoder to obtain the user representation; Leveraging user representations for recommendation prediction.

8. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of the feature-based sequence recommendation data enhancement method according to any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps in the feature-based sequence recommendation data enhancement method according to any one of claims 1 to 6 are implemented.

Images