A sample enhancement method which performs temporary operations on causal subsequences in iterations

Abstract

This invention discloses a sample augmentation method that performs temporary operations on causal subsequences during iteration. The method includes: obtaining a user set and an item set from a recommendation system; obtaining a user's historical item interaction sequence based on the user set and the item set; performing causal segmentation based on the user's historical interaction item sequence using a cross-type training sample construction strategy to obtain a causal subsequence; wherein the causal subsequence includes an input subsequence and a target subsequence; and performing temporary operations on the causal subsequence in each iteration of the training process based on a combination of random selection and probability selection swapping or deletion operations to obtain an augmented input subsequence and an augmented target subsequence. This invention proposes a non-intrusive sample augmentation technique with wide applicability and effectiveness. It leverages the contradiction between the diversity of user behavioral motivations and the uniformity of behavioral performance, performing this operation synchronously during training. By performing temporary operations on the causally segmented subsequences, it augments the training samples to address the data sparsity problem.

Description

Technical Field

[0001] This invention relates to the field of data mining technology, and more particularly to a sample augmentation method that performs temporary operations on causal sequences during iteration. Background Technology

[0002] Recommender systems aim to predict and infer users' future behavior through historical information and provide personalized services, thereby improving user experience and platform engagement. Currently, recommender systems are widely used in various scenarios (such as e-commerce, video, news content, etc.). Sequence recommendation is an important component of recommender systems; it utilizes temporal information, fully considering users' long-term and short-term preferences, to further improve the accuracy of recommendations.

[0003] Before deep learning was widely applied in recommender systems, sequence recommendation models primarily relied on Markov chains (MCs) or matrix factorization (MF) to learn the transformation relationships between items, thereby further capturing users' short-term or long-term interests. To better simulate the ternary interaction relationships (i.e., user, already interacted items, and the next item to be interacted with), translation-based models were proposed. With the introduction of deep learning networks, recurrent neural networks (RNNs) were first used to integrate temporal sequence information. Due to the vanishing gradient problem in RNN-based models, convolutional neural networks (CNNs) were considered for sequence recommendation. Since the advent of the Transformer, some models utilizing the Transformer architecture have achieved significant performance improvements. Furthermore, some recent works have constructed directed graphs by treating each user's interaction as a node, combining sequence recommendation with graph neural networks (GNNs). Therefore, existing sequence recommendation algorithms still require modifications to the internal structure or loss function of the model depending on the specific problem.

[0004] Data augmentation, as a popular technique, is widely used in various fields such as image processing and natural language processing. This invention discusses data augmentation techniques in sequence recommendation and categorizes them from two perspectives.

[0005] There are two classification perspectives: first, whether the technique only affects the data level and is unrelated to the model; and second, whether the technique introduces contextual information beyond the user and the sequence of items they interact with. Most recent work utilizes contrastive learning to augment data, designing an additional loss function in the model to bring the prediction results of similar sequences closer together while allowing the results of different sequences to diverge. Some models, such as DiffuASR, require pre-training a new model to generate new sequences, thereby augmenting the data. Both of these are considered data augmentation techniques involving model structure. Furthermore, some models augment data by utilizing additional information, such as TiCoSeRec, which uses time interval information to transform non-uniformly distributed sequences into uniformly distributed time series, and S3Rec, which utilizes item attribute information. Therefore, the form in which existing data augmentation algorithms in sequence recommendation construct samples requires adaptive operation based on the model, resulting in poor robustness.

[0006] To predict a user's future interactions, it's crucial to understand the motivations behind their past behaviors, rather than simply focusing on the surface of those behaviors. For example... Figure 2 As shown, a user who purchases a collaboration T-shirt is more likely to buy other items bearing the collaboration brand logo (even if unrelated to the T-shirt) if their motivation is the T-shirt's logo. However, if their motivation is more focused on the T-shirt itself, they are more likely to purchase other matching apparel (such as white shoes). This illustrates that even with the same interaction with the same product, subsequent behavior can differ depending on the user's reasons for choosing it. Similarly, different products can lead to similar outcomes due to similar purchase motivations. This demonstrates that while data presentation is often singular, the underlying reasons are frequently diverse and uncertain.

[0007] However, existing sequence recommendation methods rely excessively on the transitive relationships of behavioral surfaces (such as...). Figure 2 As shown by the dashed arrow in (a), repeatedly training the same item order during the training process leads to homogenization of user preferences learned by the model, which masks the rich behavioral motivations under a single behavior. Furthermore, repeatedly training on the same data increases the risk of being influenced by noise. These issues are particularly severe with sparse data.

[0008] Therefore, existing technologies still need improvement. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide a sample augmentation method that performs temporary operations on causal sequences during iteration, in order to address the technical problem that user preferences learned by existing sequence recommendation methods tend to be homogeneous under sparse data, in order to solve the technical problem that existing sequence recommendation methods learn user preferences tend to be homogeneous.

[0010] The technical solution adopted by this invention to solve the technical problem is as follows:

[0011] In a first aspect, the present invention provides a sample augmentation method for performing temporary operations on causal subsequences during iteration, comprising:

[0012] Obtain the user set and item set in the recommendation system, and obtain the user's historical item interaction sequence based on the user set and the item set;

[0013] Based on the user's historical interaction item sequence, a cross-type training sample construction strategy is used to perform causal segmentation to obtain causal subsequences; wherein, the causal subsequences include: input subsequences and target subsequences;

[0014] By combining random selection and probabilistic selection in exchange or deletion operations, temporary operations are performed on the causal subsequence in each iteration of the training process to obtain the enhanced input subsequence and the enhanced target subsequence.

[0015] In one implementation, obtaining the user set and item set in the recommendation system, and obtaining the user's historical item interaction sequence based on the user set and the item set, includes:

[0016] Obtain the user set and item set from the recommendation system;

[0017] Based on the user set, find the item sequence corresponding to each user, and associate the item sequences corresponding to each user in the user set in chronological order according to the search results to obtain the user's historical item interaction sequence.

[0018] In one implementation, the step of performing causal segmentation based on the user's historical interaction item sequence using a cross-type training sample construction strategy to obtain causal subsequences includes:

[0019] For the interaction sequence of user u The interaction sequence is divided into two parts: and Wherein, the input subsequence S input Contains the original sequence S u The target subsequence S contains all items except the last one. target Contains the original sequence S u All items except the first one.

[0020] In one implementation, the swapping or deletion operation based on a combination of random and probabilistic selection is used to perform temporary operations on the causal subsequence in each iteration of the training process to obtain the enhanced input subsequence and the enhanced target subsequence, including:

[0021] In each iteration of the training process, the probability function of the swap is determined and an item is randomly selected as the anchor item. The farthest swap distance is determined based on the first hyperparameter and the anchor item. Then, combining the probability function, the anchor item, and the farthest swap distance, a second item is probabilistically selected to swap with the anchor item, resulting in a new sequence after the swap operation.

[0022] Alternatively, in each iteration of the training process, the maximum number of items that can be deleted is determined according to the second hyperparameter, and the number of items to be deleted and the items to be deleted are randomly selected based on the maximum number of items that can be deleted. The subsequence is then deleted according to the items to be deleted to obtain a new sequence after the deletion operation.

[0023] Based on the swap operation or the deletion operation, temporary operations are performed on the input subsequence and the target subsequence respectively in each iteration of the training process to obtain the enhanced input subsequence and the enhanced target subsequence;

[0024] The enhanced training sample set is obtained by accumulating the new causal sequence samples obtained in each iteration during multiple iterations of the model training process.

[0025] In one implementation, the step of determining the furthest exchange distance based on the first hyperparameter and the anchor item, and then combining the probability function, the anchor item, and the furthest exchange distance to probabilistically select a second item to exchange with the anchor item, includes:

[0026] For a sequence S = {s1, s2, ..., s...} i ,···,s j ,···,s n Randomly select an anchor item s i ;

[0027] Based on the first hyperparameter and the probability of determining the item range using the anchor item, another neighboring item s is selected. j Wherein, the first hyperparameter is scope∈[0,1];

[0028] The anchor item s i The position and another neighboring item s selected by the probability function j Swap them to get a new sequence S′={s1,s2,···,s j ,···,s i ,···,sn}:

[0029] i = Random(1:n)

[0030]

[0031] The colon indicates the range of choices, and f(·) represents the probability function for selecting the item's location.

[0032] Based on the first hyperparameter, the neighboring items s are limited. j The percentage of the position range relative to the entire sequence length is used to determine the farthest position of the two swapped items.

[0033] In one implementation, the step of determining the maximum number of items that can be deleted based on a second hyperparameter, randomly selecting the number of items to be deleted and the items to be deleted based on the maximum number of items that can be deleted, and performing deletion operations on the subsequences according to the items to be deleted, includes:

[0034] The maximum number of items that can be deleted is determined based on the second hyperparameter;

[0035] Based on the maximum number of items that can be deleted, randomly select the number of items to be deleted and perform a corresponding number of deletion operations on the causal subsequence:

[0036]

[0037] S (t) =remove (t) (S (t-1) i t ),t∈{1,2,…,T}

[0038] i t =Random(1:|S (t-1) |)

[0039] Among them, S (t) For the sequence after deleting the t-th item, in S (t-1) Select to delete the i-th t There are 1 item; if T = 0, then S′ = S, where ρ represents the second hyperparameter.

[0040] In one implementation, the step of performing temporary operations on the causal subsequence in each iteration of the training process based on the exchange or deletion operation combining random selection and probabilistic selection to obtain the enhanced input subsequence and the enhanced target subsequence further includes:

[0041] Padding is performed on the beginning of the enhanced causal subsequence of length less than L to achieve a maximum length of L:

[0042] S={s1,s2,…,s |S| }→S′={ <pad> ,…, <pad>,s1,…,s |S| }

[0043] The length of the processed sequence is L, i.e., |S′|=L.

[0044] Secondly, the present invention provides a sample enhancement apparatus for performing temporary operations on causal sequences during iteration, comprising:

[0045] The acquisition module is used to acquire the user set and item set in the recommendation system, and obtain the user's historical item interaction sequence based on the user set and the item set;

[0046] The causal segmentation module is used to perform causal segmentation based on the user's historical interaction item sequence using a cross-type training sample construction strategy to obtain causal subsequences; wherein, the causal subsequences include: input subsequences and target subsequences;

[0047] The sample augmentation module is used to perform temporary operations on the causal subsequence in each iteration of the training process based on a combination of random selection and probability selection swapping or deletion operations, and accumulates multiple iterations to obtain the augmented input subsequence and the augmented target subsequence.

[0048] Thirdly, the present invention provides a terminal, comprising: a processor and a memory, the memory storing a sample augmentation program for performing temporary operations on causal subsequences during iteration, the sample augmentation program for performing temporary operations on causal subsequences during iteration being executed by the processor to implement the sample augmentation method for performing temporary operations on causal subsequences during iteration as described in the first aspect.

[0049] Fourthly, the present invention also provides a medium, which is a computer-readable storage medium storing a sample augmentation program that performs temporary operations on causal subsequences during iteration. When executed by a processor, the sample augmentation program that performs temporary operations on causal subsequences during iteration is used to implement the sample augmentation method for performing temporary operations on causal subsequences during iteration as described in the first aspect.

[0050] The present invention, by employing the above technical solution, has the following effects:

[0051] This invention obtains a user's historical item interaction sequence, and based on this sequence, performs causal segmentation using a cross-type training sample construction strategy to obtain causal subsequences. Furthermore, it performs temporary operations on the causal subsequences in each iteration of the training process, combining random selection and probability selection through exchange or deletion operations, to obtain enhanced input and target subsequences. This invention proposes a non-intrusive sample augmentation technique with wide applicability and effectiveness. It leverages the contradiction between the diversity of user behavioral motivations and the uniformity of behavioral performance, performing this technique synchronously during training. By performing temporary operations on the causally segmented subsequences, it enhances the training samples, thereby addressing the data sparsity problem. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0053] Figure 1 This is a flowchart of the sample augmentation method in this invention, which performs temporary operations on causal sequences during iteration.

[0054] Figure 2 This is an example diagram illustrating the relationship between user behavior and motivation.

[0055] Figure 3 This is a sample enhancement framework diagram in this invention.

[0056] Figure 4 This is a performance trend graph of the model under the first hyperparameter in this invention.

[0057] Figure 5 This is a performance trend graph of the model under the second hyperparameter in this invention.

[0058] Figure 6 This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0059] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0061] Exemplary methods

[0062] Existing sequence recommendation methods rely excessively on the transitive relationships of behavioral surfaces (e.g.) Figure 2 As shown by the dashed arrow in (a), repeatedly training the same item order during the training process leads to homogenization of user preferences learned by the model, which masks the rich behavioral motivations under a single behavior. Furthermore, repeatedly training on the same data increases the risk of being influenced by noise. These issues are particularly severe with sparse data.

[0063] To address the above-mentioned technical problems, this invention provides a sample augmentation method that performs temporary operations on causal subsequences during iteration. This method obtains a user's historical item interaction sequence and, based on this sequence, performs causal segmentation using a cross-type training sample construction strategy to obtain causal subsequences. Furthermore, it performs temporary operations on the causal subsequences in each iteration of the training process based on a combination of random selection and probability selection exchange or deletion operations, resulting in augmented input and target subsequences. Therefore, this invention proposes a non-intrusive sample augmentation technique with broad applicability and effectiveness. It leverages the contradiction between the diversity of user behavioral motivations and the uniformity of behavioral performance, performing this process synchronously during training. By performing temporary operations on the causally segmented subsequences, it augments the training samples to solve the data sparsity problem.

[0064] In this embodiment, a non-intrusive method is used to improve the performance of various sequence models without modifying the internal structure or loss function of the model. Unlike existing data classification methods, the sample augmentation method in this embodiment, which performs temporary operations on causal subsequences during iteration, operates only at the data level and is independent of the model, utilizing only the most basic information (i.e., the user and their interactive item sequence). Under this classification, this embodiment provides more training opportunities for the skeleton model compared to other methods, while also exhibiting greater stability.

[0065] like Figure 1 As shown, this embodiment of the invention provides a sample augmentation method for performing temporary operations on causal subsequences during iteration, comprising the following steps:

[0066] Step S100: Obtain the user set and item set in the recommendation system, and obtain the user's historical item interaction sequence based on the user set and the item set.

[0067] This embodiment starts with the definition of a typical sequence recommendation problem: A recommendation system has two sets: a user set U and an item set I. Each user has a sequence of items from their historical interactions, denoted as...

[0068] Based on the above information, this embodiment aims to predict the item each user will interact with next. Here, this embodiment proposes a sample enrichment via temporary operations on causal subsequences (hereinafter referred to as SETO) method during iterative training. This method leverages the contradiction between the diversity of behavioral motivations and the uniformity of behavioral performance to provide the model with more training opportunities.

[0069] Specifically, in one implementation of this embodiment, acquiring multi-source heterogeneous data includes the following steps:

[0070] Step S101: Obtain the user set and item set in the recommendation system;

[0071] Step S102: Find the item sequence corresponding to each user according to the user set, and associate the item sequences corresponding to each user in the user set in chronological order according to the search results to obtain the user's historical item interaction sequence.

[0072] In this embodiment, a user set and an item set are obtained from an existing recommendation system. Then, for each user in the user set, the user's historical clicks and purchased item sequences are searched, and the item sequences corresponding to each user in the user set are associated in chronological order to obtain the user's historical item interaction sequence.

[0073] Based on the position of the item and its actual role in the sequence, this embodiment proposes the following three reasonable assumptions, which are then used to design two different atomic operations and enhance the formation of rich training samples in each iteration of training.

[0074] Assumption A1: Behavioral motivations often possess invariance and non-uniformity. This embodiment assumes the following two scenarios: Minor changes to the input subsequence may not affect the final output. Furthermore, in reality, the same input may correspond to multiple similar results, not just a single fixed result. Note that due to language habits and grammatical rules, the tolerance in item sequences in typical recommendation systems is usually much higher than the tolerance in word sequences in natural language processing.

[0075] Assumption A2: There may be no strict order relationship between some items or subsequences in a sequence. For example, when purchasing electronic products, users are not much different in their likelihood of buying an iPhone or an iWatch first.

[0076] Assumption A3: In a long sequence, only a portion of the items may actually have a significant impact on the user's next action. Therefore, temporarily deleting some unimportant items will have little impact on the next action.

[0077] Next, the technical processing procedure of this embodiment will be described in detail. This embodiment describes the key steps in three steps: sequence causal segmentation, two temporary atomic operations, and short sequence filling.

[0078] like Figure 1 As shown, in one implementation of this invention, the sample augmentation method for performing temporary operations on causal subsequences during iteration further includes the following steps:

[0079] Step S200: Based on the user's historical interaction item sequence, a cross-type training sample construction strategy is used to perform causal segmentation to obtain causal sub-sequences.

[0080] In this embodiment, sequence causal segmentation refers to performing causal segmentation on the user's historical item interaction sequence using a cross-type training sample construction strategy to obtain causal subsequences; wherein, the causal subsequences include: the model's input subsequence and the target subsequence.

[0081] Specifically, in one implementation of this embodiment, step S200 includes the following steps:

[0082] Step S201, for the interaction sequence of user u The interaction sequence is divided into two parts: and Wherein, the input subsequence S input Contains the original sequence S u The target subsequence S contains all items except the last one. target Contains the original sequence S u All items except the first one.

[0083] In this embodiment, a cross-type training sample construction method is used as the causal segmentation method for the user's historical interaction item sequence. This is because the method can ensure that the information of the items in the sequence and the transmission relationship between the previous item and the next item are preserved to the maximum extent in the two sub-sequences after segmentation.

[0084] Cross-tabulation methods are primarily applied to models with a sequence-to-sequence (Seq2Seq) input-output format, which is already widely used in sequence models. It takes all terms except the last one as the input subsequence and all terms except the first one as the target subsequence. It appears to be a cross-tabulation structure, but differs from the segmentation methods mentioned above in that the input and target subsequences overlap, each encompassing more information.

[0085] In this embodiment, the specific causal segmentation process is as follows: for the interaction sequence of user u Divide it into two parts: and Here, the input subsequence S input Contains the original sequence S u All items except the last one, and the target subsequence S target Contains the original sequence S u All items except the first one.

[0086] In the training process of a sequence-to-sequence original model, this embodiment will use S input As input, the aim is to train the model to generate S target Due to standardization requirements, this embodiment also defines the maximum sequence length L. If the original sequence S... u If the length of the original sequence S is greater than L, this embodiment follows most previous practices and trims it to a length of L. However, if the original sequence S... u Since the length is already less than L, this embodiment will not modify it for the time being. The following description presents this step in a formulaic manner, where S represents a normal sequence, and CP(·) represents the causal segmentation operation of S.

[0087] S input ,S target =CP (S |S|-L+1:|S| (1)

[0088] Where |S|-L+1:|S| indicates that in this embodiment, only the last L items of the original sequence are taken. If the length of the original sequence is less than L, the entire original sequence S is used directly.

[0089] In this embodiment, a cross-type method is used in the causal segmentation part. Because it has overlapping parts, it can provide more training opportunities than the segmentation method and is also more in line with the semantics of predicting the next item than the masking method.

[0090] like Figure 1 As shown, in one implementation of this invention, the sample augmentation method for performing temporary operations on causal subsequences during iteration further includes the following steps:

[0091] Step S300: Based on the exchange or deletion operation combining random selection and probability selection, temporary operations are performed on the causal subsequence in each iteration of the training process to obtain the enhanced input subsequence and the enhanced target subsequence.

[0092] In this embodiment, during model training, temporary operations are performed on the causal segmented subsequences to enhance the training samples and address the data sparsity problem. This technique briefly and individually performs relevant sequence operations on the segmented causal subsequences in each repeated training iteration, which can increase the number of training samples that cannot be constructed simply by modifying the original sequence.

[0093] Regarding the temporary sequence operations performed, this embodiment fully considers the position of items in the sequence and the influence of the items themselves on the sequence, proposing two atomic operations as two variants: "Swap" or "Removal". Note that this is not merely a simple swap and removal operation. To more closely resemble real-world situations, this embodiment combines probabilistic and random selection with reasonable range constraints to enhance a practical and more diverse set of training samples, giving the model more learning space. This is an innovative sample augmentation technique that can be widely applied to skeleton sequence recommendation models using various technologies and datasets of different sizes (including industrial datasets with tens of millions of records).

[0094] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0095] Step S301: In each iteration of the training process, determine the probability function of the swap and randomly select an item as the anchor item. Determine the farthest swap distance based on the first hyperparameter and the anchor item. Then, combine the probability function, the anchor item and the farthest swap distance to probabilistically select a second item to swap with the anchor item, and obtain a new sequence after the swap operation.

[0096] In this embodiment, the causal subsequence is temporarily manipulated by either of two temporary atomic operations (i.e., swapping or deleting) to obtain the enhanced input subsequence and the enhanced target subsequence.

[0097] Based on the three assumptions above, in order to generate richer training samples, this embodiment focuses on two subsequences (i.e., S). input and S target The temporary application design incorporates two atomic operations: "Swap" and / or "Removal". To ensure the diversity and rationality of the enhanced new samples, these atomic operations incorporate random and probabilistic selection.

[0098] For the implementation of probabilistic selection, this embodiment first defines f(n,i) as the probability function. This function returns a probability value, where i represents the relative position of the item within a total length of n. The smaller the value of i, the higher the probability value. The probability values ​​for each position increase or decrease sequentially, but the sum is 1. In fact, many functions can satisfy this requirement; the following example uses one such probability function:

[0099]

[0100] The formula uses exponentiation to achieve increasing or decreasing characteristics, and α∈[0,1] is a parameter that determines the probability of selecting nearby items.

[0101] Next, we will introduce the two atomic operations designed in this embodiment, where S is regarded as the regular sequence being processed, n is its length, and S′ represents the new sample sequence constructed after processing by these operations.

[0102] Specifically, in one implementation of this embodiment, step S301 includes the following steps:

[0103] Step S301a, for a sequence S = {s1, s2, ..., s...} i ,···,s j ,···,s n Randomly select an anchor item s i ;

[0104] Step S301b: Select another neighboring item s based on the probability of the item range determined by the first hyperparameter and the anchor item. j Wherein, the first hyperparameter is scope∈[0,1];

[0105] Step S301c, the anchor item s i The position and another neighboring item s selected by the probability function j Swap them to get a new sequence S′={s1,s2,···,s j ,···,s i ,···,s n };

[0106] Step S301d: Limit the neighboring items s based on the first hyperparameter. j The percentage of the position range relative to the entire sequence length is used to determine the farthest position of the two swapped items.

[0107] In this embodiment, the specific process of the "swap" operation is as follows:

[0108] Based on assumptions A1 and A2, for a sequence S = {s1, s2, ..., s...} i ,···,s j ,···,s n In this embodiment, an anchor item s is first randomly selected. i For the second item s j For the location selection, this embodiment sets a hyperparameter scope∈[0,1] (i.e., the first hyperparameter), which limits the percentage of the location range to the entire sequence length, thus limiting the farthest position of two exchanged items. However, this embodiment still retains space for random selection when choosing items within the limited range. Then, the position of the anchor item is compared with another neighboring item s selected within the limited range by the probability function in Equation 2. j By swapping these elements, we obtain a new sequence S′={s1,s2,···,s j ,···,s i ,···,s n };

[0109] As an example, in this embodiment, Random(:) and Prob(:,f) are used to represent the random selection and probabilistic selection operations in sequence S, respectively, as shown below:

[0110] i = Random(1:n) (3)

[0111]

[0112] The colon indicates the range of choices, and f(·) represents the probability function of selecting the item position from formula (2). The hyperparameter scope ensures a certain degree of rationality for the "swap" atomic operation in this embodiment. For example, swapping a recently purchased item with an item purchased ten years ago would be unreasonable. Setting the hyperparameter as a percentage rather than a fixed value allows it to be applied to sequences of different lengths.

[0113] Specifically, in one implementation of this embodiment, step S300 further includes the following steps:

[0114] Step S302, or in each iteration of the training process, determine the maximum number of items that can be deleted according to the second hyperparameter, and randomly select the number of items to be deleted and the items to be deleted based on the maximum number of items that can be deleted, and perform deletion operations on the subsequences according to the items to be deleted to obtain a new sequence after the deletion operation;

[0115] In this embodiment, the "removal" operation refers to temporarily and randomly removing some items in each iteration of training to obtain a new sample sequence.

[0116] Specifically, in one implementation of this embodiment, step S302 includes the following steps:

[0117] Step S302a: Determine the maximum number of items that can be deleted based on the second hyperparameter;

[0118] Step S302b: Randomly select the number of items to be deleted and the items to be deleted according to the maximum number of items that can be deleted, and perform the corresponding number of deletion operations on the causal subsequence.

[0119] In this embodiment, based on assumptions A1 and A3, for the sequence S = {s1, s2, ..., s...} n In each iteration of training, some items can be temporarily and randomly removed to obtain a new sample sequence.

[0120] In each new iteration, this embodiment randomly selects the number of items to be deleted and which specific items to delete. For the number of items to be deleted, this embodiment also uses the hyperparameter ρ (i.e., the second hyperparameter) to represent the maximum percentage of the total number of items in the sequence that will be deleted. For example, if ρ×n=3, then {0,1,2,3} items in the sequence can be randomly selected for deletion. The "removal" operation allows the model to learn more about the relationships between two items at different time intervals.

[0121] It's worth noting that if an item is deleted here, in this embodiment, all other items preceding the deleted item are shifted to the right without any spaces or padding. Existing sequence recommendation models only learn the relative position of items in the sequence when learning the representation embeddings corresponding to items, without utilizing absolute time. In this embodiment, a randomly selected number T represents the number of deletion operations to be performed on sequence S, as shown in the following formula:

[0122]

[0123] S (t) =remove (t) (S (t-1) i t ),t∈{1,2,…,T} (6)

[0124] i t =Random(1:|S (t-1) |) (7)

[0125] Where S (t) It is the sequence after deleting the t-th item, in S (t-1) Select to delete the i-th t There are 10 items. If T = 0, then S′ = S.

[0126] In this embodiment, the two atomic operations, "Swap" and "Removal," maintain the probability of the original sequence remaining unchanged. Furthermore, these operations are only effective for the current iteration of training; the sequence reverts to its original form in the next iteration. Therefore, subsequent models can fully utilize the newly constructed, abundant samples during iterations. Due to their temporality and richness, even if the changed parts generate noise, the impact on the learning and training of subsequent models is minimal.

[0127] Specifically, in one implementation of this embodiment, step S300 further includes the following steps:

[0128] Step S303: Based on the swap operation or the deletion operation, temporary operations are performed on the input subsequence and the target subsequence in each iteration of the training process to obtain the enhanced input subsequence and the enhanced target subsequence;

[0129] Step S304: Based on the new causal sequence samples obtained in each iteration, the enhanced training sample set is accumulated in multiple iterations of the model training process.

[0130] In this embodiment, after the new sequence is subjected to two atomic operations, "swap" or "removal", it is divided into enhanced input subsequence and enhanced target subsequence. These enhanced subsequences can be used as training samples in the modeling process. The subsequent model can make full use of the rich samples newly constructed in the iteration.

[0131] Specifically, in one implementation of this embodiment, step S300 further includes the following steps:

[0132] Step S305: Padding is performed on the beginning of the enhanced causal sequence with a length less than L to achieve a maximum length of L.

[0133] In this embodiment, sequences shorter than L are not processed, and the length of the processed sequence is also reduced during the "removal" atomic operation. For computational convenience during model training, it is necessary to maintain a consistent sample sequence length. Padding is performed on the beginning of the sample sequence to achieve a maximum length of L, as follows:

[0134] S = {s1, s2, ..., s} |s| }→S′={ <pad> ,…, <pad>,s1,…,s |S| }

[0135] The length of the processed sequence is L, i.e., |S′|=L. Finally, the input subsequence and target subsequence processed by the above steps can be used as training samples for subsequent modeling.

[0136] Based on the above explanation of technical methods, such as Figure 3 As shown, Figure 3 The framework flowchart of this invention is provided. This invention will analyze the characteristics and advantages of the two atomic operations, "Swap" and "Removal," from different perspectives. The advantage of "Removal" over "Swap" is that it can maintain the relative position of the items that have not been deleted in the sequence in a single direction, so it is more suitable for datasets with strong sequence characteristics, such as datasets involving geographical location information.

[0137] Compared to directly deleting items from the original sequence, deleting items from the causal segmented subsequence does not cause the item to completely disappear from the entire training sample. This invention preserves item information within the sequence while constructing diverse training samples. Furthermore, the "Swap" operation does not lose any item information, thus achieving superior results when most items in the user's sequence significantly influence their preference learning. Because it does not reduce the number of items in the sequence, the "Swap" operation can construct more new training samples than "Removal." After causal segmentation, operating on the input and target subsequences separately can construct more training samples than processing only the original sequence. These new training samples are a reasonable enhancement based on the contradiction between the diversity of user behavioral motivations and the uniformity of behavioral performance; this is the key technological innovation of this invention.

[0138] To verify the technical effectiveness of this invention, three real-world practical datasets are used: Foursquare, Games, and Beauty. Foursquare is a dataset from the social application of the same name, recording check-in data from different users in different locations. This invention utilizes its historical data to predict the next location each user will check in to. It should be noted that compared to other e-commerce datasets, this dataset exhibits stronger sequential characteristics due to its inclusion of geographic location information. Games and Beauty both come from the well-known e-commerce platform Amazon. They record user reviews of games and beauty products, respectively; therefore, this invention predicts the next item a user might review based on their historical review records. These datasets are highly sparsity and have been widely used in previous work. The processing of these three datasets in this invention is consistent with FISSA.

[0139] Because this invention is non-invasive and widely applicable, it was experimentally validated by combining it with five representative and state-of-the-art sequence recommendation models (Caser, SR-GNN, SASRec, FISSA, and FMLP-Rec). Furthermore, the experimental validation highlights the advancement of this invention compared to similar RSS models, which employ a segmented sample construction method. Although it is noted that model-level data augmentation techniques differ from non-invasive techniques that operate only at the data level, this invention still underwent experimental comparison with the state-of-the-art model, the DuoRec model. It proposes a dropout-based model-level augmentation and a novel sampling strategy (i.e., sequences with the same target item are considered positive samples).

[0140] Table 1 shows the application effects of the present invention in a variety of representative skeleton models, where the present invention is represented by the abbreviation SETO, including two variants (i.e., SETO(S) and SETO(R)) using different atomic operations.

[0141] This embodiment bolds the best results and emphasizes the suboptimal results. DuoRec here improves upon SASRec, but it is not a non-intrusive method and is only used for reference.

[0142] Table 1. Performance comparison of the present invention on five representative skeleton models (full candidate set)

[0143]

[0144] Comparing the recommendation performance of the skeleton models themselves, the session-based model SR-GNN performs best on datasets with shorter sequences (i.e., Foursquare and Beauty). In the Beauty dataset, SASRec performs better. FISSA performs poorly here, presumably because it is one of the few models that incorporates candidate items into the model training, and therefore does not show its advantage on the full candidate set. To more fully demonstrate the effectiveness of this invention, experiments were conducted following FISSA and supplemented with partial candidate set experiments as shown in Table 2.

[0145] Table 2. Performance comparison of the present invention in models SASRec and FISSA (partial candidate set)

[0146]

[0147]

[0148] Overall, this invention significantly improves the recommendation performance of all five backbone models. Sequence-to-sequence models perform better than sequence-to-item models. For the FMLP-Rec model with denoising, it partially offsets our enhancements, so the improvement is not significant, but still present. RSS improves the performance of the backbone model on denser datasets (i.e., Foursquare), but it is not stable on sparse datasets. As for the model-level DuoRec, it enhances SASRec on the Foursquare and Games datasets, but its performance is inferior to this invention. To adapt to different datasets and models, this invention designs two different temporary atomic operations: "Swap" and "Removal". "Removal" is more suitable for datasets more sensitive to the order of items in the sequence, such as Foursquare, because this operation does not change the order of items in the sample sequence. In sparse datasets, "Swap" enriches the sample types, as seen in the performance of FMLP-Rec on the Games and Beauty datasets.

[0149] This invention cleverly processes the subsequence after causal segmentation, rather than the conventional approach of only processing the original sequence. The invention further verifies the effectiveness of this step through experiments. The experiments are applied to the SASRec model in three different scenarios (i.e., processing the input subsequence S alone). i Process the target subsequence S separately t Process both subsequences S i+t The results are shown in Table 3.

[0150] Table 3. Performance Comparison of the Invention on Different Subsequences

[0151]

[0152] Clearly, this invention achieves significant performance improvements whether applied to a single subsequence or both subsequences simultaneously. For "Swap," experiments show that processing both the input and target subsequences simultaneously yields better results than processing them individually. This is because simultaneous processing provides a richer variety of training samples. However, for "Removal," simultaneous processing is less effective than processing a single subsequence. This is because deleting only one item does not cause the deleted item to completely disappear from the pair of training samples, but applying deletion to both the input and target subsequences may result in the loss of information about some items, despite randomization. This experimental verification demonstrates that both atomic operations designed in this invention perform well and even better under different applicable conditions.

[0153] The combination of random selection and probabilistic selection, along with a reasonable range limitation, is one of the innovations of the two atomic operations in this invention. This invention is also verified experimentally, and the results are as follows: Figure 4 and Figure 5 As shown.

[0154] like Figure 4 As shown, Figure 4 This is a performance trend graph of the model under the hyperparameter scope. Figure 4 (a) shows the performance trend of the Foursquare model. Figure 4 (b) is the performance trend chart of the Games model. Figure 4 (c) shows the performance trend of the Beauty model. Compared to fixed values, these two hyperparameters were designed as percentages in this invention, which better adapts to subsequences of different lengths. For the hyperparameter `scope`, datasets with more short sequences, i.e., Foursquare, show better recommendation performance with larger hyperparameter values, while datasets like Games and Beauty perform well with a hyperparameter of 0.5. This indicates that setting a hyperparameter to specify the furthest position of the "swap" operation is reasonable.

[0155] like Figure 5 As shown, Figure 5 This is a performance trend graph of the model under hyperparameter ρ. Figure 5 (a) shows the performance trend of the Foursquare model. Figure 5 (b) is the performance trend chart of the Games model. Figure 5 Figure (c) shows the performance trend of the Beauty model. The hyperparameter ρ determines the maximum number of items to be removed from the subsequence. All three datasets show a similar trend: initially increasing, then decreasing. This is because excessively large ρ values ​​remove too much information about the items, while excessively small values ​​are not conducive to generating rich samples. In summary, by selecting appropriate hyperparameter values ​​based on the sequence information of each dataset, this invention can make the augmented samples under these temporary operations closer to the reality.

[0156] To verify that this invention, as a non-intrusive data augmentation technique, can enrich training samples and overcome the challenge of data sparsity, it was applied to a large-scale industrial dataset. This industrial dataset consists of user listening logs from a popular music streaming platform. Specifically, the experiment used samples from the past 10 days for training and the data from the second day for testing, thus the training set and test set contained approximately 80 million and 8 million data samples, respectively. The model Base is a deployment sequence recommendation method based on SASRec. Based on industry practice, the effectiveness of this invention as a non-intrusive data augmentation technique was evaluated using Recall@50 and Recall@100. The experimental results are shown in Table 4.

[0157] Table 4 Performance Comparison of the Invention on Large-Scale Industrial Datasets

[0158] Models Recall@50 Recall@100 Base 0％ 0％ Base_SETO(S) 1.19％ 1.12％ Base_SETO(R) 1.16％ 0.73％

[0159] The results show that both variants of this invention can improve the Recall@50 metric by 1.1% compared to the base model. In this large-scale dataset of tens of millions of samples, the improvement is very significant, and this invention does not make any changes to the model architecture or loss function.

[0160] The extensive experiments demonstrated above prove that the present invention remains effective and widely applicable under various architectures and in datasets of varying sizes.

[0161] This embodiment achieves the following technical effects through the above technical solution:

[0162] This embodiment fully considers the position of items in the sequence and the influence of the items themselves on the sequence. It proposes two atomic operations as two variants, which briefly and independently perform relevant sequence operations on the segmented causal subsequences in each repeated training iteration. This can increase the number of training samples that cannot be constructed by simply changing the original sequence. Furthermore, by combining probabilistic selection and random selection with reasonable range constraints, a set of practical and more diverse training samples is enhanced, giving the model more learning space. This embodiment proposes a non-intrusive sample augmentation technique with wide applicability and effectiveness. It takes advantage of the contradiction between the diversity of user behavior motivations and the uniformity of behavior performance, and performs it synchronously during the training process. By performing temporary operations on the causal segmented subsequences, it enhances the training samples to solve the data sparsity problem.

[0163] Exemplary device

[0164] Based on the above embodiments, the present invention also provides a sample enhancement apparatus for performing temporary operations on causal sequences during iteration, comprising:

[0165] The acquisition module is used to acquire the user set and item set in the recommendation system, and obtain the user's historical item interaction sequence based on the user set and the item set;

[0166] The causal segmentation module is used to perform causal segmentation based on the user's historical interaction item sequence using a cross-type training sample construction strategy to obtain causal subsequences; wherein, the causal subsequences include: input subsequences and target subsequences;

[0167] The sample augmentation module is used to perform temporary operations on the causal subsequence in each iteration of the training process based on a combination of random selection and probability selection swapping or deletion operations, and accumulates multiple iterations to obtain the augmented input subsequence and the augmented target subsequence.

[0168] This embodiment achieves the following technical effects through the above technical solution:

[0169] This embodiment fully considers the position of items in the sequence and the influence of the items themselves on the sequence. It proposes two atomic operations as two variants, which briefly and independently perform relevant sequence operations on the segmented causal subsequences in each repeated training iteration. This can increase the number of training samples that cannot be constructed by simply changing the original sequence. Furthermore, by combining probabilistic selection and random selection with reasonable range constraints, a set of practical and more diverse training samples is enhanced, giving the model more learning space. This embodiment proposes a non-intrusive sample augmentation technique with wide applicability and effectiveness. It takes advantage of the contradiction between the diversity of user behavior motivations and the uniformity of behavior performance, and performs it synchronously during the training process. By performing temporary operations on the causal segmented subsequences, it enhances the training samples to solve the data sparsity problem.

[0170] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 6 As shown.

[0171] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a storage medium and internal memory; the storage medium stores the operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external devices; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or other devices.

[0172] When executed by a processor, this computer program is used to implement a sample augmentation method that performs temporary operations on causal sequences during iteration.

[0173] It will be understood by those skilled in the art that Figure 6 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0174] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a sample augmentation program that performs temporary operations on causal subsequences during iteration, the sample augmentation program that performs temporary operations on causal subsequences during iteration being executed by the processor to implement the above-described sample augmentation method for performing temporary operations on causal subsequences during iteration.

[0175] In one embodiment, a storage medium is provided, wherein the storage medium stores a sample augmentation program that performs temporary operations on causal subsequences during iteration, the sample augmentation program that performs temporary operations on causal subsequences during iteration being executed by a processor to implement the above-described sample augmentation method for performing temporary operations on causal subsequences during iteration.

[0176] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, database, or other media used in the embodiments provided by this invention can include both non-volatile and volatile memory.

[0177] In summary, this invention provides a sample augmentation method that performs temporary operations on causal subsequences during iteration. The method includes: obtaining a user set and an item set from a recommendation system; obtaining a user's historical item interaction sequence based on the user set and the item set; performing causal segmentation based on the user's historical interaction item sequence using a cross-type training sample construction strategy to obtain a causal subsequence; wherein the causal subsequence includes an input subsequence and a target subsequence; and performing temporary operations on the causal subsequence in each iteration of the training process based on a combination of random selection and probability selection swapping or deletion operations to obtain an augmented input subsequence and an augmented target subsequence. This invention proposes a non-intrusive sample augmentation technique with wide applicability and effectiveness. It utilizes the contradiction between the diversity of user behavioral motivations and the uniformity of behavioral performance, performing the augmentation synchronously during training by performing temporary operations on the causally segmented subsequences to augment the training samples, thereby solving the data sparsity problem.

[0178] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.< / pad> < / pad> < / pad> < / pad>

Claims

1. A sample augmentation method that performs temporary operations on causal subsequences during iteration, characterized in that, include: Obtain the user set and item set in the recommendation system, and obtain the user's historical item interaction sequence based on the user set and the item set; Based on the user's historical interaction item sequence, a cross-type training sample construction strategy is used for causal segmentation to obtain causal subsequences, including: for users Interaction sequence The interaction sequence is divided into two parts: and ; where, input subsequence Contains the original sequence All items except the last one, target subsequence Contains the original sequence All items except the first item; wherein the causal subsequence includes: an input subsequence and a target subsequence; In each iteration of the training process, temporary operations are performed on the causal subsequence based on swap or deletion operations to obtain an enhanced input subsequence and an enhanced target subsequence. This includes: in each iteration of the training process, determining a probability function for swapping and randomly selecting an item as an anchor item; determining the furthest swap distance based on a first hyperparameter and the anchor item; then, combining the probability function, the anchor item, and the furthest swap distance, probabilistically selecting a second item to swap with the anchor item to obtain a new sequence after the swap operation; or in each iteration of the training process, determining the maximum number of items that can be deleted based on a second hyperparameter, and based on the maximum number of items that can be deleted... The number of items to be deleted is randomly selected, and the number of items to be deleted is determined. Deletion operations are then performed on the subsequences based on the items to be deleted, resulting in new sequences after the deletion operations. Based on the exchange or deletion operation, temporary operations are performed on the input and target subsequences in each iteration of the training process to obtain the enhanced input and target subsequences. The enhanced training sample set is accumulated over multiple iterations of the model training process based on the new causal subsequence samples obtained in each iteration. The exchange or deletion operation is a combination of random and probabilistic selection.

2. The sample augmentation method for performing temporary operations on causal subsequences during iteration according to claim 1, characterized in that, The process of obtaining the user set and item set in the recommendation system, and obtaining the user's historical item interaction sequence based on the user set and the item set, includes: Obtain the user set and item set from the recommendation system; Based on the user set, find the item sequence corresponding to each user, and associate the item sequences corresponding to each user in the user set in chronological order according to the search results to obtain the user's historical item interaction sequence.

3. The sample augmentation method for performing temporary operations on causal subsequences during iteration according to claim 1, characterized in that, The step of determining the furthest exchange distance based on the first hyperparameter and the anchor item, and then combining the probability function, the anchor item, and the furthest exchange distance to probabilistically select a second item to exchange with the anchor item includes: For a sequence Randomly select an anchor item ; Based on the first hyperparameter and the probability of determining the item range using the anchor item, another neighboring item is selected. Wherein, the first hyperparameter is ; The anchor item The location of another neighboring item selected by the probability function Swap to get a new sequence : The colon indicates the range of choices available. The probability function representing the selection of an item's location; The neighboring items are limited based on the first hyperparameter. The percentage of the position range relative to the entire sequence length is used to determine the farthest position of the two swapped items.

4. The sample augmentation method for performing temporary operations on causal subsequences during iteration according to claim 1, characterized in that, The process of determining the maximum number of items that can be deleted based on the second hyperparameter, randomly selecting the number of items to be deleted and the items to be deleted based on the maximum number of items that can be deleted, and performing deletion operations on the subsequences according to the items to be deleted, includes: The maximum number of items that can be deleted is determined based on the second hyperparameter; Based on the maximum number of items that can be deleted, randomly select the number of items to be deleted and perform a corresponding number of deletion operations on the causal subsequence: in, To delete the first The sequence following each item, in Select to delete the One item; if ,but , This represents the second hyperparameter.

5. The sample augmentation method for performing temporary operations on causal subsequences during iteration according to claim 1, characterized in that, Also includes: Padding is performed on the beginning of the enhanced causal subsequence of length less than L to achieve the maximum length. : The length of the processed sequence is ,Right now .

6. A sample augmentation apparatus for performing temporary operations on causal subsequences during iteration, used to implement the sample augmentation method for performing temporary operations on causal subsequences during iteration as described in any one of claims 1-5, characterized in that, include: The acquisition module is used to acquire the user set and item set in the recommendation system, and obtain the user's historical item interaction sequence based on the user set and the item set; The causal segmentation module is used to perform causal segmentation based on the user's historical interaction item sequence using a cross-type training sample construction strategy to obtain causal subsequences; wherein, the causal subsequences include: input subsequences and target subsequences; The sample augmentation module is used to perform temporary operations on the causal subsequence in each iteration of the training process based on a combination of random selection and probability selection swapping or deletion operations, and accumulates multiple iterations to obtain the augmented input subsequence and the augmented target subsequence.

7. A terminal, characterized in that, include: The processor and memory, the memory storing a sample augmentation program that performs temporary operations on causal subsequences during iteration, the sample augmentation program that performs temporary operations on causal subsequences during iteration being executed by the processor to implement the sample augmentation method for performing temporary operations on causal subsequences during iteration as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a sample augmentation program that performs temporary operations on causal subsequences during iteration. When executed by a processor, the sample augmentation program that performs temporary operations on causal subsequences during iteration is used to implement the sample augmentation method for performing temporary operations on causal subsequences during iteration as described in any one of claims 1-5.

Images