Method for generating neural network for item recommendation, method for item recommendation and device thereof
By introducing a neural network with a frequency ramp sampling layer, a time-domain self-attention layer, and a frequency-domain attention layer into the e-commerce system, the problem of insufficient accuracy in item recommendation in existing technologies is solved, and a more efficient item recommendation effect is achieved.
Patent Information
- Application Number
- CN202310237936.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-13
- Publication Date
- 2025-12-05
- Estimated Expiration
- 2043-03-13
AI Technical Summary
Existing e-commerce systems struggle to effectively utilize users' historical interaction data, especially early interaction data, when making product recommendations, resulting in insufficient recommendation accuracy.
A neural network generation method is adopted, including a frequency ramp sampling layer, a temporal self-attention layer, and a frequency domain attention layer. By training the neural network, the feature extraction and pattern recognition capabilities of user interaction data are enhanced, and a neural network capable of item recommendation is generated.
It improves the accuracy and efficiency of item recommendations, enabling better prediction of items that users may visit or purchase in the future, thereby increasing the likelihood of user purchases.
Smart Images
Figure CN116228365B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of item recommendation, and in particular to a method for generating a neural network for item recommendation, an item recommendation method and device thereof. BACKGROUND
[0002] With the progress of society and the development of information technology, more and more users use e-commerce systems for online shopping, online ordering of takeout, online ordering of movie tickets, and so on. When a user visits, purchases, or evaluates an item (which can be a product, takeout, movie ticket, etc.), the e-commerce system stores the interaction data. It can be understood that if the e-commerce system can process the interaction data and then obtain a number of items that the user is likely to visit or purchase in the future, and then recommend the number of items to the user, the likelihood of the user purchasing the item can be greatly improved. SUMMARY
[0003] Therefore, the main purpose of the present application is to provide a method for generating a neural network for item recommendation, an item recommendation method and device thereof.
[0004] To achieve the above purpose, the technical solution of the present application is as follows: a method for generating a neural network for item recommendation, comprising the following steps: obtaining a plurality of interaction queues, each interaction queue corresponding to a user, each interaction queue containing a number of items interacted by the corresponding user, and the earlier the occurrence of the interaction of the item, the closer to the head of the interaction queue; creating a neural network with attention mechanism, the neural network comprising a frequency slope sampling layer, a time domain self-attention layer and a frequency domain attention layer; training the neural network based on all interaction queues.
[0005] As an improvement of an embodiment of the present application, the frequency slope sampling layer is configured to: obtain the encoding corresponding to all items in each interaction queue and generate an interaction queue E; in the pth layer along the item dimension, the input time feature is , and the output time feature is , and is the input of the p+1th layer, and the input of the first layer self-attention module ; the pth layer is further configured to perform , and F() is one-dimensional fast Fourier transform; wherein p is a natural number, 1≤p≤the item dimension of the neural network.
[0006] As an improvement of an embodiment of the present application, the time domain self-attention layer is configured to: linearly project each of the encodings corresponding to all items in the interaction queue E , thereby obtaining , and , and then the following processing is performed: , , , wherein, is a sampling function, padding() is a padding function, is an inverse Fourier transform function, softmax() is an activation function, is a sampling ratio, and D is a number value of the item dimension.
[0007] As an improvement of the embodiment of the present application, the frequency domain attention layer is used for: delaying the interaction queue E and obtaining an interaction queue For each encoding in the interaction queue , linear projection is performed, so as to obtain , and ; and then the following processing is performed: , , wherein, padding() is a padding function, is an inverse Fourier transform function, is a sampling ratio, and * is a conjugate operation of a complex number, is element-wise multiplication.
[0008] As an improvement of the embodiment of the present application, the frequency domain attention layer is further used for: generating The attention weight of different sequences is calculated as: wherein, N is the length of the interaction queue , Topk() is a function of taking the maximum k, and k is a natural number; ; , wherein, and represent unsupervised and supervised incremental views respectively, , is a weight value, and N is the length of the input interaction queue .
[0009] As an improvement of the embodiment of the present application, the training of the neural network based on all interaction queues specifically includes: training the neural network based on all interaction queues, and when training, the overall objective function is wherein and are hyperparameters.
[0010] As an improvement of the embodiment of the present application, the recommendation probability corresponding to the item is , the objective function is , wherein, is an item ID embedding matrix composed of all different items in all interaction queues, and V is a set composed of all different items in all interaction queues, is the number of items in the set v, is the recommendation probability corresponding to the i-th item in the set V, i is a natural number, 1≤i≤ .
[0011] The embodiment of the application also provides a neural network generation device for item recommendation, comprising the following modules: an information acquisition module, configured to acquire a plurality of interaction queues, each interaction queue corresponding to a user, each interaction queue containing a plurality of items interacted by the corresponding user, and the earlier the occurrence event of the interaction corresponding to the item, the closer to the head of the interaction queue; a model creation module, configured to create a neural network of attention mechanism, the neural network comprising a frequency slope sampling layer, a time domain self-attention layer and a frequency domain attention layer; and a training module, configured to train the neural network based on all interaction queues.
[0012] The embodiment of the application also provides an item recommendation method, comprising the following steps: executing the generation method and obtaining a neural network; acquiring a plurality of items interacted by a preset user, inputting the plurality of items into the neural network, and obtaining recommended items.
[0013] The embodiment of the application also provides an item recommendation device, comprising the following modules: a preprocessing module, configured to execute the generation method and obtain a neural network; and a processing module, configured to acquire a plurality of items interacted by a preset user, input the plurality of items into the neural network, and obtain recommended items.
[0014] The item recommendation neural network generation method, item recommendation method and device provided by the embodiment of the application have the following advantages: the embodiment of the application discloses an item recommendation neural network generation method, item recommendation method and device, the generation method comprising: acquiring a plurality of interaction queues, each interaction queue corresponding to a user, each interaction queue containing a plurality of items interacted by the corresponding user, and the earlier the occurrence event of the interaction corresponding to the item, the closer to the head of the interaction queue; creating a neural network of attention mechanism, the neural network comprising a frequency slope sampling layer, a time domain self-attention layer and a frequency domain attention layer; and training the neural network based on all interaction queues. Thus, a neural network capable of item recommendation is generated. BRIEF DESCRIPTION OF DRAWINGS
[0015] Figure 1 FIG. 1 is a flowchart of the neural network generation method in the embodiment.
[0016] Figure 2 and Figure 3 Figures of experimental results for the neural network in the examples. DETAILED DESCRIPTION
[0017] The present application will be described in detail below with reference to the embodiments shown in the drawings. However, the embodiments do not limit the present application, and modifications of structure, method, or function made by those skilled in the art based on the embodiments are included in the scope of the present application.
[0018] The following description and drawings are illustrative of the specific embodiments herein and are not intended to limit the present application. Parts and features of some embodiments can be included or substituted in or for parts and features of other embodiments. The scope of the embodiments herein includes the full scope of the claims and all available equivalents of the claims. In this document, the terms "first," "second," and so on do not necessarily mean "one," "two," and so on, but are used to distinguish one element from another. In fact, a first element can also be referred to as a second element, and vice versa. Also, the terms "comprise," "include," or their other variants are intended to encompass non-exclusive inclusion, such that a structure, device, or apparatus that comprises, includes, or is included in, one list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such structure, device, or apparatus. Without more limitation, an element defined by the phrase "comprising a... " does not exclude the existence of additional identical elements in the structure, device, or apparatus that includes the element. Various embodiments are described in a progressive manner, each focusing on the differences from other embodiments, and the same or similar parts between embodiments can be referred to each other.
[0019] The terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like in the present document indicate the orientation or positional relationship shown in the drawings, and are used only for convenience of description and simplification of the description herein, and do not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In the description herein, unless otherwise specified and limited, the terms "mounting," "connection," and "connection" should be broadly understood, for example, can be mechanical connection or electrical connection, can be internal communication of two elements, can be direct connection, or indirect connection through an intermediate medium, and the specific meaning of the above terms can be understood by those skilled in the art according to the specific circumstances.
[0020] Embodiment 1 of the present invention provides a method for generating a neural network for item recommendation, such as... Figure 1 As shown, it includes the following steps:
[0021] Step 101: Obtain multiple interaction queues. Each interaction queue corresponds to a user. Each interaction queue contains several items that the corresponding user has interacted with. The earlier the interaction event for an item occurs, the closer it is to the head of the interaction queue.
[0022] Here, all users can be extracted from all interaction queues, thus obtaining the user set. This user set The number of users included is Furthermore, all items can be extracted from all interaction queues, thus obtaining an item set. This collection of items The number of items included is Construct the interaction sequence of user u. ,in, Indicates user The t-th item has been interacted with (e.g., purchased, visited, rated, etc.), and N represents the length of the interaction sequence. It can be understood that, given a user-item interaction sequence... The goal of this generation method is to generate a neural network that can predict the next most likely item that user u will click. .
[0023] In practice, different items can be assigned different IDs, and then an item ID embedding matrix can be created. By converting the item ID into an embedding vector through lookup, a high-dimensional one-hot encoding is projected onto a low-dimensional dense representation, where d is the embedding size (i.e., the length of the one-hot encoding). The user's behavior sequence is encoded using the following formula: The LookUp(.,.) function retrieves the embedding vector from the embedding matrix. Input: User and item IDs; Output: Embedded representations of users and items; Implementation: Create an embedding vector table and query for user and item IDs.
[0024] Step 102: Create a neural network with an attention mechanism, the neural network comprising a frequency ramp sampling layer, a temporal self-attention layer, and a frequency domain attention layer;
[0025] In this neural network, the feature extraction capability of the neural network in the local frequency band is first enhanced by adding a frequency sampling module. Secondly, in order to make full use of the complex features in the frequency domain, the time domain self-attention layer and the frequency domain attention layer are combined to form a new hybrid attention layer.
[0026] Step 103: training the neural network based on all interaction queues.
[0027] In this embodiment, the frequency ramp sampling layer is configured to: obtain the encoding corresponding to all items in each interaction queue and generate an interaction queue E; along the p-th layer in the item dimension, the input time feature is , the output time feature is , and is the input of the p+1-th layer, the input of the first self-attention module ; the p-th layer is further configured to perform , F() is one-dimensional fast Fourier transform; wherein p is a natural number, 1≤p≤the item dimension of the neural network.
[0028] Here, in the p-th self-attention module, F is first performed, wherein is the time feature of the p-th layer. It can be understood that the entire neural network is stacked by N self-attention modules, is the input of the first self-attention module, and the output of the first self-attention module is , is the input of the second self-attention module, and the output of the second self-attention module is ,... is the input of the p-th self-attention module, and the output of the p-th self-attention module is .
[0029] Here, due to the conjugate symmetry property in the frequency domain, half of the DFT (Discrete Fourier Transformation) contains complete information about the periodicity in the time domain. M is calculated as follows: , The sequence length of is almost equal to half of . It can be understood that is a complex tensor representing the spectrum of
[0030] In this embodiment, the time domain self-attention layer is configured to: perform linear projection on each of the encodings in the encodings corresponding to all items in the interaction queue E , thereby obtaining , and , and then perform the following processing: , , , , wherein is a padding function, is an inverse Fourier transform function, and softmax() is an activation function, is a sampling ratio, and D is a number of item dimensions.
[0031] Here, for the interaction queue E, after linear projection, the following is obtained , and are converted into frequency spectrums by FFT (Fast Fourier Transform), and a sampling layer is performed on each frequency component. In addition, before calculating the attention weight in the time domain, the sampling frequency needs to be zero-padded to the original shape. In summary, the domain self-attention can not only learn the low-frequency information of the top layer, but also learn the high-frequency information of the bottom layer, thereby improving the ability of the neural network to capture local behavior.
[0032] In the embodiment, the frequency domain attention layer is configured to perform delay processing on the interaction queue E and obtain an interaction queue For each encoding in the interaction queue , linear projection is performed to obtain , and ; then the following processing is performed: , , where padding() is a padding function, is an inverse Fourier transform function, is a sampling ratio, and * is a conjugate operation of a complex number, is an element-wise multiplication.
[0033] Here, the existing self-attention sequence recommender tends to capture the global view of user interaction at the item level, but lacks the periodic similar behavior pattern captured at the sequence level. According to the Wiener-Khinchin theorem, by calculating the autocorrelation, the most relevant time delay sequence can be found in the frequency domain, thereby discovering the periodicity hidden in the behavior. Specifically, given the interaction queue , its time delay operation can be expressed as: .
[0034] In the embodiment, the frequency domain attention layer is further configured to generate , and the attention weight of different sequences is calculated as: where N is the length of the interaction queue , Topk() is a function of taking the maximum k, and k is a natural number; ; , where, and represent unsupervised and supervised incremental view respectively, , is the weight value, N is the length of the input interaction queue .
[0035] Here, the sampled feature can be represented as where, is the shape after sampling. Before performing the inverse Fourier transform, the sampled frequency needs to be zero-padded to the original shape. The autocorrelation is calculated based on the Wiener-Khinchin theorem. Autocorrelation refers to the correlation of a time series with its own past and future, and then the topk most relevant ones are selected from it. Finally, the output of the frequency domain attention module and the output of the time domain attention module are added with the hyperparameter ratio to obtain The implementation process can be understood as follows: using a sliding filter mixer to capture different frequency patterns of the dynamic sequence, the weights of the filter are adaptively adjusted according to the input data, and the sliding filter mixer further includes two frequency division strategies, dynamic frequency selection and static frequency division module. After stacking multiple layers of sliding filter mixer layers, random dropout is used to prevent overfitting, and normalization is used to improve training speed and stability, and finally the final user and item vector representation is obtained.
[0036] In this embodiment, the "training the neural network based on all interaction queues" specifically includes:
[0037] Based on all interaction queues, the neural network is trained, and when training, the overall objective function is where, and are hyperparameters.
[0038] In this embodiment, the recommendation probability corresponding to the item is , and the objective function is where, is the item ID embedding matrix composed of all different items in all interaction queues, V is a set composed of all different items in all interaction queues, is the number of items in the set v, is the recommendation probability corresponding to the i-th item in the set V, i is a natural number, and 1≤i≤ .
[0039] To evaluate the performance of the neural network, the inventors conduct experiments on four widely used datasets for recommendation. The datasets are Amazon Beauty, Amazon Clothing, Amazon Sports, ML-1M, and Yelp, respectively, and the detailed data of the datasets are shown in Table 1. Figure 2
[0040] The inventors adopt two most commonly used performance evaluation indicators, i.e., recall and normalized discounted cumulative gain (NDCG). The recall rate is the accuracy of ground-truth items appearing in the top N recommendations. The NDCG is a ranking loss used to measure the position of the true items in the top N recommendations. For each user, the items are sorted in descending order according to the predicted scores to generate a recommendation list, and the top N items are truncated to calculate the top N results of the evaluation indicators. Since Recall@1 is equivalent to NDCG@1, the results of Recall@{5, 10, 20} and NDCG@{5, 10, 20} are reported. In addition, the inventors compare with common sequential recommendation models, and the specific recommendation performance is shown in Table 2. Figure 3
[0041] Embodiment two of the present application provides a neural network for generating an article recommendation device, comprising the following modules:
[0042] An information acquisition module is configured to acquire a plurality of interaction queues, each of which corresponds to a user, and each of which contains a plurality of articles interacted by the corresponding user, and the earlier the occurrence of the interaction of the article, the closer to the head of the interaction queue.
[0043] A model creation module is configured to create a neural network with an attention mechanism, which includes a frequency ramp sampling layer, a time domain self-attention layer, and a frequency domain attention layer.
[0044] A training module is configured to train the neural network based on all interaction queues.
[0045] Embodiment three of the present application provides an article recommendation method, comprising the following steps: performing the generation method described in embodiment one to obtain a neural network; acquiring a plurality of articles that have interacted with a preset user, and inputting the plurality of articles into the neural network to obtain recommended articles.
[0046] Embodiment four of the present application provides an article recommendation device, comprising the following modules: a preprocessing module configured to perform the generation method described in embodiment one to obtain a neural network; and a processing module configured to acquire a plurality of articles that have interacted with a preset user, and input the plurality of articles into the neural network to obtain recommended articles.
[0047] It is to be understood that even though various steps of the method are described in a particular order, the method can be performed in a different order, or some of the steps can be performed concurrently, or even changed, so long as the desired function is performed.
[0048] The present application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.
[0049] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves.
[0050] Embodiments of the application have been described above, and the description is not exhaustive, and does not limit the scope of the disclosed embodiments. Many modifications and variations of the described embodiments will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The description is intended to cover any and all modifications and variations, including such concepts contained therein of which the author is not explicitly aware. The terms used herein are to be construed in their broadest, ordinary sense, and are not to be limited by any slight changes that can be made to the specification.
Claims
1. A method for generating a neural network for item recommendation, characterized in that, The method comprises the following steps: Obtaining a plurality of interaction queues, each of which corresponds to a user, and each of which contains a plurality of items interacted by the corresponding user, and the earlier the occurrence event of the interaction corresponding to the item, the closer to the head of the interaction queue; A neural network with an attention mechanism is created, comprising a frequency ramp sampling layer, a temporal self-attention layer, and a frequency domain attention layer. The frequency ramp sampling layer is used to: acquire the codes corresponding to all items in each interaction queue and generate an interaction queue E; along the p-th layer of the item dimension, its input temporal features are... Its output time characteristics are ,and This is the input to the (p+1)th layer, and the input to the self-attention module of the first layer. The p-th layer is also used for execution. F() is a one-dimensional Fast Fourier Transform; where p is a natural number, 1 ≤ p ≤ the item dimension of the neural network; the temporal self-attention layer is used for: for each code in the encoding corresponding to all items in the interaction queue E Linear projection is performed on all of them to obtain , and Then, the following processing is performed: , , , ,in, It's the sampling function, and `padding()` is the padding function. It is the inverse Fourier transform function, and softmax() is the activation function. This represents the sampling ratio, where D is the number of item dimensions; the frequency domain attention layer is used to: perform delayed processing on the interaction queue E and obtain the interaction queue. For interactive queues Each code in Linear projection is performed on all of them to obtain , and Then, the following processing is performed: , , Here, `padding()` is the padding function. It is the inverse Fourier transform function. This represents the sampling ratio, and * represents the conjugate operation for complex numbers. It is element-wise multiplication; generation The attention weights for different sequences are calculated as follows: Where N is the interaction queue The length of the string, Topk() is a function that takes the k largest elements, where k is a natural number; ; , ,in, and These represent unsupervised and supervised incremental views, respectively. , is a weight value, N is the length of the input interaction queue ; Based on all the interaction queues, the neural network is trained, and during the training, the overall objective function is wherein and are hyperparameters; the recommendation probability corresponding to the item is and the objective function is wherein, is an item ID embedding matrix composed of all different items in all interaction queues, V is a set composed of all different items in all interaction queues, is the number of items in the set v, is the recommendation probability corresponding to the i-th item in the set V, i is a natural number, and 1≤i≤ .
2. A device for generating a neural network for item recommendation, characterized in that, The method comprises the following modules: An information acquisition module is configured to obtain a plurality of interaction queues, each of which corresponds to a user, and each of which contains a plurality of items interacted by the corresponding user, and the earlier the occurrence event of the interaction corresponding to the item, the closer to the head of the interaction queue; The model creating module is configured to create a neural network of an attention mechanism, which includes a frequency slope sampling layer, a time domain self-attention layer, and a frequency domain attention layer. The frequency slope sampling layer is configured to obtain encoding corresponding to all items in each interaction queue and generate an interaction queue E. In the pth layer along the item dimension, the time feature of the input is , and the time feature of the output is , and is the input of the (p+1)th layer and the input of the first self-attention module ; the pth layer is further configured to perform , and F() is a one-dimensional fast Fourier transform. Wherein, p is a natural number, 1≤p≤the item dimension of the neural network; the time domain self-attention layer is configured to perform linear projection on each encoding in the encoding corresponding to all items in the interaction queue E, thereby obtaining , , , and . Then, the following processing is performed: , , , , wherein, is a sampling function, padding() is a padding function, is an inverse Fourier transform function, softmax() is an activation function, is a sampling ratio, and D is a numerical value of the item dimension; the frequency domain attention layer is configured to perform delay processing on the interaction queue E and obtain an interaction queue . Linear projection is performed on each encoding in the interaction queue , thereby obtaining , , and . Then, the following processing is performed: , , , wherein, padding() is a padding function, is an inverse Fourier transform function, is a sampling ratio, * is a complex conjugate operation, is element-wise multiplication; and is generated. The attention weight of different sequences is calculated as: , wherein N is the length of the interaction queue , Topk() is a function of taking the maximum k, and k is a natural number. ; , , wherein, and represent unsupervised and supervised incremental views, respectively. , is a weight value, N is the length of the input interaction queue ; The training module is configured to train the neural network based on all the interaction queues, and during the training, a total objective function is wherein and are hyperparameters; a recommendation probability corresponding to the item is and the objective function is wherein, is an item ID embedding matrix composed of all different items in all the interaction queues, V is a set composed of all different items in all the interaction queues, is the number of items in the set v, is a recommendation probability corresponding to the i-th item in the set V, i is a natural number, and 1≤i≤ .
3. An article recommendation method characterized by comprising: The method comprises the following steps: Executing the generation method of claim 1 to obtain a neural network; Obtaining a plurality of items interacted by a preset user, inputting the plurality of items into the neural network, and obtaining recommended items.
4. An article recommendation device characterized by comprising: The method comprises the following modules: A preprocessing module is configured to execute the generation method of claim 1 to obtain a neural network; A processing module is configured to obtain a plurality of items interacted by a preset user, input the plurality of items into the neural network, and obtain recommended items.