Deformable attention sequence recommendation method and system based on progressive sampling

Abstract

The application provides a deformable attention sequence recommendation method and system based on progressive sampling. The method comprises the following steps: obtaining a historical interaction sequence of a target user interacting with an item at different times, and performing standardization processing on the historical interaction sequence to generate a plurality of embedded sequences containing item and position information; determining a plurality of sampling reference points in each embedded sequence, and generating a sampling point at each time step of each embedded sequence through a lightweight sampling network for position and feature; updating the sampling point through a plurality of preset deformable attention progressive sampling DAPS modules, including sequentially and gradually refining the sampling point through each DAPS module; optimizing and training a recommendation model based on the step-by-step refinement of the sampling point, and outputting a recommendation result for the target user through the trained recommendation model. The method uses a deformable attention model based on progressive sampling to perform sequence recommendation, thereby improving the flexibility, efficiency and accuracy of sequence recommendation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a deformable attention sequence recommendation method and system based on progressive sampling. BACKGROUND

[0002] At present, sequence recommendation has been widely applied in the field of item recommendation. Sequence recommendation is a recommendation system paradigm that models user behavior and item patterns in time series to recommend related items to users. It can be applied to various applications, such as recommending possible purchases to users on e-commerce websites.

[0003] In related technologies, when performing sequence recommendation, a neural network model based on RNN, CNN or Transformer is usually used to implement sequence recommendation. However, the above related sequence recommendation scheme may introduce some associated noise, which reduces the accuracy of the recommendation results output by the recommendation model, and the recommendation results deviate greatly from the real interests of users. Moreover, when the amount of data processed in actual application is large, the model training process is difficult and consumes a large amount of time and space resources.

[0004] Therefore, how to more conveniently generate recommendation results that are more consistent with the real interests of users through sequence recommendation has become a problem to be solved at present. SUMMARY

[0005] The present application aims to at least solve one of the technical problems in the related art to some extent.

[0006] To this end, a first object of the present application is to propose a deformable attention sequence recommendation method based on progressive sampling, which samples key-value pairs from sequences in a variable manner to establish a sparse attention mechanism suitable for each query and gradually improves the sampling strategy across layers to reduce the difficulty of learning and improve the flexibility, efficiency and accuracy of sequence recommendation.

[0007] A second object of the present application is to propose a deformable attention sequence recommendation system based on progressive sampling.

[0008] A third object of the present application is to propose a non-transitory computer-readable storage medium.

[0009] To achieve the above object, a first aspect of the present application proposes a deformable attention sequence recommendation method based on progressive sampling, comprising the following steps:

[0010] Obtaining the historical interaction sequence of the target user interacting with the item at different times, and performing standardization processing on the historical interaction sequence to generate a plurality of embedded sequences containing item and position information;

[0011] a plurality of sampling reference points are determined in each of the embedding sequences, and based on the sampling reference points, sampling points are generated at each time step of each of the embedding sequences by a lightweight sampling network for positions and features;

[0012] The sampling points are updated by a plurality of preset variable attention progressive sampling DAPS modules, including sequentially refining the sampling points by each of the DAPS modules step by step to improve the sampling strategy.

[0013] The recommendation model is optimized and trained based on the step-by-step refinement of the sampling points, and a recommendation result for the target user is output by the trained recommendation model.

[0014] Optionally, in an embodiment of the present application, the standardization processing of the historical interaction sequence to generate a plurality of embedding sequences containing item and position information includes: truncating the historical interaction sequence into a plurality of sequences according to a fixed length, and aligning the truncated plurality of sequences; embedding item information and position information in the aligned sequence based on an embedding matrix to generate an input sequence containing item embedding and position embedding; and performing overfitting prevention processing and layer normalization processing on the input sequence to generate the embedding sequence.

[0015] Optionally, in an embodiment of the present application, the sampling reference points are determined by the following formula:

[0016] S′ (i，j) = n' + i * π j / k, i ∈ 1,..., k

[0017] Wherein, π j = j - n',

[0018] Wherein, S′ (i，j) represents the i-th sampling reference point at the j-th time step, n' represents the first non-zero position of any embedding sequence, π j represents the meaningful length in the query sequence at the j-th time step, and k represents the number of sampling reference points.

[0019] Optionally, in an embodiment of the present application, the lightweight sampling network for positions and features generates sampling points at each time step of each of the embedding sequences, including: predicting the offset of the sampling points according to the sequence features of any embedding sequence by two aggregation layers in the lightweight sampling network, and compressing and extracting global sequence information by a channel-based global aggregation layer to generate the offset corresponding to the any embedding sequence; and calculating the offset points by adding the offset according to the element arrangement to the corresponding sampling reference points, and taking the offset points at each time step as the sampling points generated at the corresponding time step.

[0020] Optionally, in an embodiment of the present application, the step-by-step refinement of the sampling points by each DAPS module comprises: after determining the initial set of sampling points, each DAPS module takes the sampling points output by the previous adjacent DAPS module as the input sampling points of the current DAPS module according to the arrangement order; each DAPS module learns an offset according to its own query feature, and each DAPS module adds the offset to the corresponding input sampling points to obtain updated sampling points.

[0021] Optionally, in an embodiment of the present application, the step-by-step refinement of the sampling points is used to optimize and train the recommendation model, which comprises: taking the embedding sequence as the input of the recommendation model, taking the offset sequence obtained in the step-by-step refinement of the sampling points as the output of the recommendation model, and setting the loss function in the training process of the recommendation model as the negative log-likelihood function of the reference label ground-truth.

[0022] To achieve the above purpose, the second aspect embodiment of the present application proposes a deformable attention sequence recommendation system based on progressive sampling, which comprises the following modules:

[0023] An acquisition module is configured to acquire a historical interaction sequence of a target user interacting with an item at different times, and perform standardization processing on the historical interaction sequence to generate a plurality of embedding sequences containing item and location information;

[0024] A generation module is configured to determine a plurality of sampling reference points in each embedding sequence, and generate sampling points at each time step of each embedding sequence based on the sampling reference points through a lightweight sampling network for location and feature;

[0025] An update module is configured to update the sampling points through a plurality of deformable attention progressive sampling (DAPS) modules, including step-by-step refinement of the sampling points by each DAPS module to improve the sampling strategy;

[0026] An output module is configured to optimize and train a recommendation model based on the step-by-step refinement of the sampling points, and output a recommendation result for the target user through the trained recommendation model.

[0027] Optionally, in an embodiment of the present application, the acquisition module is specifically configured to: truncate the historical interaction sequence into a plurality of sequences according to a fixed length, and align the truncated plurality of sequences; embed item information and location information in the aligned sequences based on an embedding matrix to generate an input sequence containing item embedding and location embedding; and perform overfitting prevention processing and layer normalization processing on the input sequence to generate the embedding sequence.

[0028] Optionally, in an embodiment of the present application, the generating module is specifically configured to: predict the offset of a sampling point according to sequence features of any embedding sequence through two aggregation layers in the lightweight sampling network, and compress and extract global sequence information through a channel-based global aggregation layer to generate the offset corresponding to the any embedding sequence; and calculate the offset point by adding the offset and the corresponding sampling reference point according to the element arrangement, and take the offset point at each time step as the sampling point generated at the corresponding time step.

[0029] To achieve the above-mentioned embodiments, the third aspect of the present application further proposes a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the deformable attention sequence recommendation method based on progressive sampling in the above-mentioned embodiments.

[0030] The technical scheme provided by the embodiments of the present application at least brings the following beneficial effects: the present application samples key-value pairs from sequences in a variable manner to establish a sparse attention mechanism suitable for each query, and gradually improves the sampling strategy across layers to reduce the difficulty of learning. The present application uses a new deformable attention model to model the current dynamics in the sequence, where the interaction items of each query are selected in a data-dependent manner, so that the query can pay more attention to relevant behaviors and provide more context information to predict the next item. Thus, based on the progressive sampling strategy, the present application enhances the recommendation model, making the recommendation model more flexible and efficient, capable of capturing more information features and eliminating noise correlation. Therefore, the present application makes the sequence recommendation result more matched with the real interest of the user, reduces the resources consumed by model training, reduces the recommendation cost, and improves the accuracy of sequence recommendation.

[0031] Additional aspects and advantages of the present application will be made apparent by the following description and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0032] The above-mentioned and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0033] Figure 1 A flowchart of a deformable attention sequence recommendation method based on progressive sampling is proposed for the embodiments of the present application;

[0034] Figure 2 A flowchart of a standardization processing method of an interaction sequence is proposed for the embodiments of the present application;

[0035] Figure 3 A schematic diagram of the working principle of a DAPS module is proposed for the embodiments of the present application;

[0036] Figure 4 A flow architecture diagram of a deformable attention sequence recommendation based on progressive sampling proposed for an embodiment of the present application;

[0037] Figure 5 A flowchart of a specific deformable attention sequence recommendation method based on progressive sampling proposed for an embodiment of the present application;

[0038] Figure 6 A structure diagram of a deformable attention sequence recommendation system based on progressive sampling proposed for an embodiment of the present application. DETAILED DESCRIPTION

[0039] Embodiments of the present application are described in detail below with reference to examples illustrated in the accompanying drawings, in which the same or similar components or components having the same or similar functions are denoted by the same or similar reference numerals throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0040] It should be noted that sequence recommendation is a recommendation algorithm that represents users through single behavior sequence mining, and the recommendation system arranges these interaction behaviors in time sequence one by one, mines the sequential patterns using various modeling methods, and is used to support the recommendation of one or more items at the next moment. The types of sequence recommendation can be divided into three categories according to the use scenario, namely, time sequence recommendation, session recommendation and time sequence dialogue recommendation. It can be applied to various recommendations such as commodities, news and videos.

[0041] In related art, sequence recommendation is usually performed based on RNN, CNN or Transformer recommendation model. For example, a neural attention recommendation machine NARM is proposed in related art to solve the shortcoming of only considering the current session, and for example, a new short-term attention priority model is proposed in some schemes to consider the user's current machine from the user's short-term memory of the last click. These methods use attention to enhance the original structure.

[0042] And because the Transformer and BERT models are different from the attention support in the past, they have self-attention, which has also been introduced into the recommendation field. For example, SASRec is a sequence model that allows capturing long-term semantics (such as RNN) and using attention mechanisms to make predictions based on relatively few actions (such as MC), but it is a left-to-right unidirectional model that limits the representativeness of the hidden vector. For example, BERT4Rec proposes to model user behavior sequences by using deep bidirectional self-attention to solve these limitations, which adopts the purpose of cloze test, and predicts the randomly masked items in the sequence by the joint conditioning of its left and right context.

[0043] However, the above scheme still cannot meet the current sequence recommendation demand, and there are corresponding defects. Specifically, first, the dense attention operation of the above self-attention-based model can cause noise association, including irrelevant behaviors, random clicks, and malicious forgeries, thereby limiting the ability of the model to capture the user's real interest in the sequence pattern. Second, as the input length increases, the time and space consumed are greatly expanded.

[0044] To this end, the present application proposes a deformable attention sequence recommendation method based on progressive sampling, which performs sequence recommendation based on a deformable attention model based on progressive sampling, thereby improving the flexibility, efficiency and accuracy of sequence recommendation.

[0045] The deformable attention sequence recommendation method based on progressive sampling proposed by the embodiments of the present application is described below with reference to the accompanying drawings.

[0046] Figure 1 The flowchart of the deformable attention sequence recommendation method based on progressive sampling proposed by the embodiments of the present application is shown in Figure 1 The method comprises the following steps:

[0047] Step S101, obtaining the historical interaction sequence of the target user interacting with the items at different times, and performing standardization processing on the historical interaction sequence to generate a plurality of embedded sequences containing item and position information.

[0048] Among them, the target user is a user to be recommended for item recommendation, and the target user can be one or multiple in a user set. The items interacted by the target user can be various goods with entities, etc., or various data information such as news, music and video, which are not limited here.

[0049] Specifically, the known target user interaction sequence according to time is first obtained, i.e. the historical interaction data of the target user with the items at different time points as the time changes, which can be regarded as a pre-processed input sequence. The sequence recommendation method of the present application predicts the item that the next user may interact according to the obtained historical interaction sequence, i.e. predicts the item that the user may be interested in at the next time according to the known item that the user is interested in, and recommends the predicted item to the user.

[0050] Specifically, assuming that the obtained historical interaction sequence is: Among them, u is a user set, I is an item set, and the sequence represents the historical interaction sequence of user u. The present application aims to S uBased on this, a predictive model is constructed to predict the next interactive item. During model training, in each iteration time t step, the model predicts the next item based on the previous item. That is, this application defines the model input as... The expected output is

[0051] Furthermore, when training a recommendation model based on the acquired known data to recommend items, it is necessary to determine the optimization method of the recommendation model. This application first standardizes the historical interaction sequence to generate multiple embedding sequences containing item and location information.

[0052] Specifically, the user's interaction sequence over time is standardized to generate an embedding sequence with item and location information. This embedding sequence can be used as training data for the recommendation model.

[0053] To more clearly illustrate the specific implementation process of the standardization process in this application, the following description uses an embodiment of the standardization method for historical interaction sequences as an example. Figure 2 This is a flowchart of a standardized processing method for interactive sequences proposed in an embodiment of this application, as shown below. Figure 2 As shown, the method includes the following steps:

[0054] Step S201: The historical interaction sequence is truncated into multiple sequences with a fixed length, and the truncated sequences are aligned.

[0055] Specifically, the historical interaction sequence S u Convert to a fixed-length input [s1, s2, ..., s] n ], where S u Data exceeding a fixed length n is truncated forward, while data shorter than n is padded with zeros to ensure that the resulting sequences are of equal length. These sequences are then aligned to the same position. Here, n represents the maximum length of data the prediction model can process. In one embodiment of this application, the historical interaction sequence of a user u can be truncated into multiple sequences and aligned. Alternatively, when making recommendations to multiple users, the sequences corresponding to multiple users can be aligned sequentially.

[0056] Step S202: Embed item information and location information in the aligned sequence based on the embedding matrix to generate an input sequence containing item embedding and location embedding.

[0057] Specifically, the item and location information are embedded into a sequence to form a sequence E′ containing both item and location information. E′ consists of two parts and can be represented by the following formula:

[0058]

[0059] wherein, is the embedding matrix of items, d is the latent dimension, and for input sequence features wherein, is the embedding matrix of positions,

[0060] In step S203, the input sequence is subjected to overfitting prevention processing and layer normalization processing to generate an embedding sequence.

[0061] Specifically, the input sequence E' described above is subjected to overfitting prevention processing, i.e., Dropout processing, and layer normalization processing (1a yernormalization), so as to perform random initialization of the sequence. The generated embedding sequence can finally be expressed as

[0062] It should be noted that since the generated embedding sequence is a sequence available for model learning, the embedding sequence can also be described as an input sequence of the model in the subsequent description of the present application. Since alignment of multiple sequences is performed in step S201, the above processing can be performed on each of the aligned sequences, thereby generating multiple embedding sequences.

[0063] In step S102, multiple sampling reference points are determined in each embedding sequence, and based on the sampling reference points, sampling points are generated at each time step of each embedding sequence by a lightweight sampling network for positions and features.

[0064] Specifically, before formal data sampling, the purpose of determining multiple sampling reference points in each embedding sequence is to realize initialization of the sampling points so as to stabilize the learning process. The present application can initialize the sampling points by different ways of generating unified reference points.

[0065] As one possible implementation, the sampling reference points can be determined by the following formula:

[0066] S' (i，j) = n' + i * π j / k, i∈1,..., k

[0067] wherein, π j = j - n',

[0068] wherein, S' (i，j) represents the i-th sampling reference point at the j-th time step, n' represents the first non-zero position of any embedding sequence, π jdenotes the length of the query sequence that has meaning at the j-th time step, and k denotes the number of sampled reference points. The sampled reference points can be determined in the above manner.

[0069] Further, based on the determined sampled reference points, a lightweight sampling network capable of simultaneously considering position and feature, which is designed in advance in the application, is used to generate sampling points at each time step of each embedding sequence.

[0070] In an embodiment of the application, the method of generating sampling points at each time step of each embedding sequence by a lightweight sampling network for position and feature can include the following two steps: first, the offset of the sampling point is predicted from the sequence feature of any embedding sequence by two aggregation layers in the lightweight sampling network, and the global sequence information is compressed and extracted by a channel-based global aggregation layer to generate the offset corresponding to any embedding sequence. Second, the offset point is calculated by adding the offset arranged by elements to the corresponding sampling reference point, and the offset point at each time step is taken as the sampling point generated at the corresponding time step.

[0071] Specifically, the preset lightweight sampling network predicts the offset of the sampling point from the input sequence feature by two aggregation layers; and a channel-based global aggregation layer is used to compress and extract global sequence information. The layer in the network mainly consists of two channel linear projection layers along the time dimension, which can be represented by the following formula:

[0072]

[0073] Further, the offset predicted in the first step is sorted and added to the corresponding sampling reference point determined in the above embodiment to obtain the offset point S generated by the offset prediction arranged by elements (0) , i.e. (0) The sampling reference point S' and the offset O' are composed of two parts, which can be calculated by the following formula:

[0074] S (0) = S' + O'

[0075] Wherein, the offset

[0076] Step S103, updating the sampling points by a plurality of variable attention progressive sampling DAPS modules, including sequentially refining the sampling points by each DAPS module to improve the sampling strategy.

[0077] Specifically, the application pre-designs multiple deformable attention progressive sampling (DAPS) modules, sequentially updates the sampling points generated in the previous step by a new deformable attention model, realizes step-by-step refinement of the multiple DAPS modules, thereby establishing a sparse attention mechanism suitable for each query and gradually refining and perfecting the sampling strategy.

[0078] As a possible implementation manner, the DAPS module constructed by the application is as shown in Figure 3 Figure 3 FIG. 1 is a schematic diagram of a working principle of a DAPS module according to an embodiment of the application, and the DAPS module (i.e., DAPS Block in Figure 3 ) includes a deformable-attention layer (Deformable-Attention), two Add&Norm layers and a feedforward layer (FeedForward).

[0079] Specifically, the Add&Norm includes two structures, i.e., Add and Layer Norm, the Add is mainly applied to a residual network (ResNet), and the Layer Norm is a standardization processing on each vector, for example, it can be a calculation of mean value or standard deviation. The related functions of the Add&Norm layer and the Feed Forward layer can be referred to the related art, and will not be described here. The core of the DAPS module is a deformable-attention submodule, which can effectively model the relationship between tokens under the guidance of the sampling points. The working principle of the Deformable-Attention is as shown in Figure 3

[0080] Among them, the input of each DAPS Block is the sequence feature output by the previous module (in Figure 3 , the sequence feature input for the current bth DAPS module is F (b-1) , and the sampling points updated by the previous module (i.e., S (b-1) ). The Deformable-Attention module updates the input sampling points (Sampling Points) to deformed points (Deformed Points) for the position flow, and updates the sequence feature by the sampling feature and the multi-head attention mechanism (Multi-Head Attention) for the feature flow, thereby the bth DAPS module outputs the updated sequence feature F (b) and the sampling points S (b) . ​​

[0081] That is, the DAPS block of the present application constructs fine-grained sparse attention in a data-dependent manner, simplifies dense attention, and maintains fine-grained item-by-item interaction. The present application illustrates the deformable attention operation on the time step, making it easier to understand. As shown in Figure 3 The sequence representation and the sampling point representation within each DAPS block are as shown in the following formula:

[0082]

[0083] F (b) =FFN(z (b) )

[0084] z (b) ,S (b) ←MHDA(F (b-1) ,S (b-1) )。

[0085] When the DAPS module based on the above design updates the sampling points, in the embodiment, the following steps are included: after determining the initial group of sampling points, each DAPS module takes the sampling points output by the previous adjacent DAPS module as the input sampling points of the current DAPS module according to the arrangement order; each DAPS module learns the offset according to its own query feature, and each DAPS module adds the input sampling points corresponding to the offset to obtain the updated sampling points.

[0086] Specifically, after determining the first group of sampling points, each DAPS block takes the previous group of sampling points as input, learns the offset according to the query feature of the current layer, and gradually refines the sampling position. The sampling points are updated by adding the sampling points to the offset of the current layer at each layer, which can be represented by the following formula:

[0087] S (b) =S (b-1) +O (b) ,b∈1,...,L

[0088] Thus, each deformable attention layer is referenced to the previous sampling position and repeatedly corrected to achieve the final purpose.

[0089] In step S104, the recommendation model is optimized and trained based on the step-by-step refinement of the sampling points, and the recommendation result for the target user is output by the trained recommendation model.

[0090] Specifically, based on the results obtained in the above steps, the optimization method of the recommendation model is determined, and the recommendation model is trained, and then the trained recommendation model is used to predict according to the input historical interaction sequence, thereby outputting the recommendation result of the target user.

[0091] In an embodiment of the present application, the step-by-step refinement based on sampling points is used to optimize and train the recommendation model, including: taking the embedding sequence as the input of the recommendation model, taking the offset sequence obtained in the step-by-step refinement process of the sampling points as the output of the recommendation model, and setting the loss function in the training process of the recommendation model as the negative log-likelihood function of the reference label ground-truth.

[0092] Specifically, when training the prediction model, the obtained historical sequence s = [s1, s2, …, sT] is taken as the input, the corresponding offset sequence O calculated in the above embodiment is taken as the output, and the loss function of the model training is defined as the negative log-likelihood of the ground-truth. The loss function can be represented by the following formula: n

[0093]

[0094] Further, in the embodiment, the information is further gathered step by step from the position of interest to obtain the final output F( L ) of the sequence to predict the next item. Specifically, a feedforward layer is applied between the two, to produce a targeted output distribution on the ground-truth, which can be represented by the following formula:

[0095]

[0096] Therefore, the recommendation model of the embodiment of the present application can output the recommended result according to the obtained historical interaction sequence of the target user.

[0097] It should be noted that, in order to more clearly represent the meaning of each formula or each parameter in the above embodiment, the parameters involved are described as follows. Among them, is the item embedding matrix; is the position embedding matrix; is the input sequence feature; is the input sequence with position embedding; is the sequence feature after the bth block; is the recommended point; is the sampling point after the bth block; is the offset generated by the network; is the offset generated by the bth DAPS block; is the embedding matrix of the query, key and value in the attention mechanism; is the query vector of time step t; is the query q t ​an embedding matrix of the sampled key and value; d e N is a latent vector dimension; n e N is a maximum sequence length; L e N is a number of DAPS blocks; k e N is a number of sampling points per query; and h e N is a number of heads per attention operation.

[0098] In summary, the deformable attention sequence recommendation method based on progressive sampling of the embodiments of the present application samples key-value pairs from the sequence in a variable manner to establish a sparse attention mechanism suitable for each query and gradually refines the sampling strategy across layers to reduce the difficulty of learning. The method uses a new deformable attention model to model the current dynamics in the sequence, where the interaction items of each query are selected in a data-dependent manner, enabling the query to focus more on relevant behaviors and providing more context information for predicting the next item. Thus, the method based on the progressive sampling strategy enhances the recommendation model, making the recommendation model more flexible and efficient, capable of capturing more information features and eliminating noise correlations. As a result, the method makes the sequence recommendation result more matched to the user's real interest, reduces the resources consumed by model training, reduces the recommendation cost, and improves the accuracy of sequence recommendation.

[0099] Based on the above embodiments, in order to more clearly describe the specific processing flow of the sequence recommendation method of the present application, in an embodiment of the present application, a whole architecture of a deformable attention recommendation based on progressive sampling and a specific deformable attention sequence recommendation method based on progressive sampling are exemplarily described. Figure 4 A flow architecture diagram of a deformable attention sequence recommendation based on progressive sampling is proposed for the embodiments of the present application, Figure 5 A flow diagram of a specific deformable attention sequence recommendation method based on progressive sampling is proposed for the embodiments of the present application.

[0100] As shown in the figure, Figure 4 After obtaining the input sequence (i1, i2, … i n ), the feature flow (Feature Flow) is processed through the embedding layer (Embedding Layer) and L DAPS modules, and the sequence feature is predicted in the prediction layer (Predicition Layer). For the position flow (Position Flow), the selection of the sampling reference points (reference points) is performed in sequence, the sampling points are generated through the sampling network (Sampling Netwok), and the sampling points are progressively refined (Progressively refine) through multiple DAPS modules.

[0101] As shown in the figure, Figure 5 The recommendation method includes the following steps:

[0102] Step S501, inputting a user's interaction sequence according to time.

[0103] Step S502, embedding features of items and locations.

[0104] Specifically, for the inputted user's interaction sequence according to time, vectorization processing is performed, and finally a sequence with item and location information embedding is formed for model learning.

[0105] Step S503, determining a sampling reference point.

[0106] Specifically, before formal sampling, a unified reference point is generated as an initialized sampling point to stabilize the learning process.

[0107] Step S504, generating a sampling point using a lightweight sampling network.

[0108] Specifically, according to the designed lightweight sampling network considering location and features, a sampling point is generated at each time step of each input sequence.

[0109] Step S505, gradually refining based on a DAPS block.

[0110] Specifically, according to the designed DAPS block, the sampling point is updated to establish a sparse attention mechanism suitable for each query, and the sampling strategy is gradually refined and perfected.

[0111] Step S506, result prediction.

[0112] Specifically, based on the above steps, the optimization method of the model is determined, and the user's recommendation result is output according to the training result of the model.

[0113] It should be noted that the specific implementation of each step in the method can refer to the related description of the above embodiments, which will not be repeated here.

[0114] In order to realize the above-mentioned embodiments, the present application also proposes a deformable attention sequence recommendation system based on progressive sampling. Figure 6 A structure diagram of a deformable attention sequence recommendation system based on progressive sampling is proposed for the embodiments of the present application.

[0115] As shown in the figure, the system includes an acquisition module 100, a generation module 200, an update module 300 and an output module 400. Figure 6 Among them, the acquisition module 100 is used to acquire the historical interaction sequence of the target user interacting with the item at different times, and to standardize the historical interaction sequence to generate multiple embedding sequences containing item and location information.

[0116]

[0117] ​The generating module 200 is configured to determine a plurality of sampling reference points in each embedding sequence, and generate a sampling point at each time step of each embedding sequence based on the sampling reference points through a lightweight sampling network for positions and features.

[0118] The updating module 300 is configured to update the sampling points through a plurality of variable attention progressive sampling DAPS modules, including sequentially refining the sampling points through the DAPS modules one by one to perfect the sampling strategy.

[0119] The output module 400 is configured to optimize and train a recommendation model based on the refined sampling points, and output a recommendation result for a target user through the trained recommendation model.

[0120] Optionally, in an embodiment of the present application, the acquiring module 100 is specifically configured to truncate the historical interaction sequence into a plurality of sequences according to a fixed length, and align the truncated plurality of sequences; embed item information and position information in the aligned sequence based on an embedding matrix to generate an input sequence containing item embedding and position embedding; and perform overfitting prevention processing and layer normalization processing on the input sequence to generate the embedding sequence.

[0121] Optionally, in an embodiment of the present application, the generating module 200 is specifically configured to determine the sampling reference points through the following formula:

[0122] S′ (i，j) = n' + i * π j / k, i ∈ 1,..., k

[0123] wherein π j = j - n',

[0124] wherein S′ (i，j) represents the i-th sampling reference point at the j-th time step, n' represents the first non-zero position of any embedding sequence, π j represents the length of significance in the query sequence at the j-th time step, and k represents the number of sampling reference points.

[0125] Optionally, in an embodiment of the present application, the generating module 200 is further configured to: predict the offset of the sampling point according to the sequence features of any embedding sequence through two aggregation layers in the lightweight sampling network, and compress and extract global sequence information based on a channel-based global aggregation layer to generate the offset corresponding to any embedding sequence; and calculate the offset point by adding the offset according to the element arrangement to the corresponding sampling reference point, and take the offset point at each time step as the sampling point generated at the corresponding time step.

[0126] Optionally, in an embodiment of the present application, the updating module 300 is specifically configured to: after determining the initial set of sampling points, each DAPS module takes the sampling point output by the previous adjacent DAPS module as the input sampling point of the current DAPS module according to the arrangement order; each DAPS module learns an offset according to the query feature of itself, and each DAPS module adds the corresponding input sampling point to the offset to obtain an updated sampling point.

[0127] Optionally, in an embodiment of the present application, the output module 400 is specifically configured to: take the embedding sequence as the input of the recommendation model, take the offset sequence obtained in the step-by-step refinement process of the sampling point as the output of the recommendation model, and set the loss function in the training process of the recommendation model as the negative log-likelihood function of the reference label ground-truth.

[0128] It should be noted that the foregoing description of the embodiment of the deformable attention sequence recommendation method based on progressive sampling also applies to the system of the present embodiment, and the implementation principle is the same, which will not be described here.

[0129] In summary, the deformable attention sequence recommendation system based on progressive sampling of the embodiments of the present application samples key-value pairs from the sequence in a variable manner to establish a sparse attention mechanism suitable for each query and gradually refines the sampling strategy across layers to reduce the difficulty of learning. The system uses a new deformable attention model to model the current dynamics in the sequence, where the interaction items of each query are selected in a data-dependent manner, enabling the query to focus more on relevant behaviors and providing more context information for predicting the next item. Thus, the system enhances the recommendation model based on the progressive sampling strategy, making the recommendation model more flexible and efficient, capable of capturing more information features and eliminating noise correlations. As a result, the system makes the sequence recommendation result more matched to the user's true interest, reduces the resources consumed by model training, reduces the recommendation cost, and improves the accuracy of sequence recommendation.

[0130] In order to implement the above-mentioned embodiments, the present application further proposes a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the deformable attention sequence recommendation method based on progressive sampling of the first aspect of the present application.

[0131] In the description of the application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the application. In the description of the application, if the illustrative expressions of the above terms are used in a plurality of embodiments or examples, it does not mean that the embodiments or examples are the same. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the skilled person in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples, without contradiction.

[0132] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the application, the meaning of "multiple" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0133] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing the specified logic functions or processes, and the preferred embodiments of the application also include additional implementation involving other processes or methods. The preferred embodiments of the application should be understood to include all modifications, combinations, and sub-combinations of the other implementations described herein.

[0134] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a sequence of executable instructions stored in a computer readable medium, which can be executed by an instruction execution system, apparatus or device, such as a computer-based system, a processor-based system, or other system that can fetch the instructions from the instruction execution system, apparatus or device and execute the instructions, or a combination thereof. For the purposes of this specification, a "computer readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus or device. The computer readable medium can specifically be, but is not limited to, the following: an electronic connection (electronic apparatus) having one or more wires, a portable computer diskette (magnetic apparatus), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disk read-only memory (CDROM). In addition, the computer readable medium can even be paper or other suitable medium upon which the program can be printed, because the program can be electronically obtained, for example, by optically scanning the paper or other medium, then

[0135] It should be understood that portions of the application can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, or a combination thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0136] Those of ordinary skill in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium. When the programs are executed, they include one of the steps of the method embodiments or a combination thereof.

[0137] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing module, or each of the units can be physically present separately, or two or more units can be integrated in one module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0138] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A deformable attention sequence recommendation method based on progressive sampling, characterized by, The method comprises the following steps: obtaining a historical interaction sequence of a target user interacting with an item at different times, and performing standardization processing on the historical interaction sequence to generate a plurality of embedding sequences containing item and location information; determining a plurality of sampling reference points in each embedding sequence, and predicting the offset of a sampling point from the sequence features of any embedding sequence through two aggregation layers in a lightweight sampling network based on the sampling reference points, and compressing and extracting global sequence information through a channel-based global aggregation layer to generate the offset corresponding to the any embedding sequence; calculating the offset point by adding the offset according to the element arrangement to the corresponding sampling reference point, and taking the offset point at each time step as the sampling point generated at the corresponding time step; updating the sampling point through a plurality of variable attention progressive sampling (DAPS) modules, including: after determining the initial group of sampling points, each DAPS module takes the sampling point output by the adjacent previous DAPS module as the input sampling point of the current DAPS module according to the arrangement order; each DAPS module learns an offset amount according to its own query features, and each DAPS module adds the corresponding input sampling point to its own offset amount to obtain an updated sampling point to improve the sampling strategy; wherein one DAPS module includes a variable attention layer, two residual and normalization layers, and a feedforward layer; optimizing and training the recommendation model based on the step-by-step refinement of the sampling points, and outputting a recommendation result for the target user through the trained recommendation model.

2. The deformable attention sequence recommendation method based on progressive sampling according to claim 1, characterized in that, The standardization processing of the historical interaction sequence to generate a plurality of embedding sequences containing item and location information comprises: truncating the historical interaction sequence into a plurality of sequences according to a fixed length, and aligning the truncated plurality of sequences; embedding item information and location information in the aligned sequence based on an embedding matrix to generate an input sequence containing item embedding and location embedding; performing overfitting prevention processing and layer normalization processing on the input sequence to generate the embedding sequence. 3.The deformable attention sequence recommendation method based on progressive sampling according to claim 1, wherein, The sampling reference points are determined by the following formula: wherein , in, Indicates the first The first time step One sampling reference point, This indicates the position of the first non-zero position in any embedded sequence. Indicates the first The length of the query sequence that has meaning at each time step. This indicates the number of sampling reference points.

4. The deformable attention sequence recommendation method based on progressive sampling of claim 1, wherein, The optimization and training of the recommendation model based on the step-by-step refinement of the sampling points comprises: taking the embedding sequence as the input of the recommendation model, taking the offset sequence obtained in the step-by-step refinement of the sampling points as the output of the recommendation model, and setting the loss function in the training process of the recommendation model as the negative log likelihood function of the reference label (ground-truth).

5. A deformable attention sequence recommendation system based on progressive sampling, characterized in that, It comprises: an acquisition module configured to obtain a historical interaction sequence of a target user interacting with an item at different times, and perform standardization processing on the historical interaction sequence to generate a plurality of embedding sequences containing item and location information; The generating module is configured to determine a plurality of sampling reference points in each of the embedding sequences, and based on the sampling reference points, predict offsets of sampling points according to sequence features of any embedding sequence by two aggregation layers in a lightweight sampling network, and compress and extract global sequence information by a channel-based global aggregation layer to generate offsets corresponding to the any embedding sequence; and calculate offset points by adding the offsets and corresponding sampling reference points according to element arrangement, and take the offset points at each time step as sampling points generated at the corresponding time step. The updating module is configured to update the sampling points by a plurality of variable attention progressive sampling (DAPS) modules, including: after determining an initial group of sampling points, each DAPS module takes sampling points output by a previous DAPS module as input sampling points of the current DAPS module according to arrangement order; each DAPS module learns an offset amount according to query features of the DAPS module, and each DAPS module adds the input sampling points and the offset amount to obtain updated sampling points to improve the sampling strategy; wherein one DAPS module includes a variable attention layer, two residual and normalization layers, and a feedforward layer. The output module is configured to optimize and train a recommendation model based on step-by-step refinement of the sampling points, and output a recommendation result for the target user by the trained recommendation model.

6. The progressive sampling-based deformable attention sequence recommendation system of claim 5, wherein, The obtaining module is specifically configured to: truncate the historical interaction sequence into a plurality of sequences according to a fixed length, and align the plurality of truncated sequences; embed item information and position information in the aligned sequences based on an embedding matrix to generate input sequences containing item embeddings and position embeddings; perform overfitting prevention processing and layer normalization processing on the input sequences to generate the embedding sequences.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the variable attention sequence recommendation method based on progressive sampling according to any one of claims 1-4.

Images