Information recommendation method, device and equipment
By introducing the interaction and entanglement mechanism between the encoder and the decoder into the information recommendation model of the user's preferred time series, the problem of the inability to accurately extract user interest characteristics in the prior art is solved, and a more accurate information recommendation effect is achieved.
Patent Information
- Application Number
- CN202411666580.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-20
- Publication Date
- 2025-03-11
AI Technical Summary
In the prior art, the transformer model lacks the interaction and entanglement between the encoder and the decoder, which leads to the inability to accurately extract user interest characteristics, thereby affecting the accuracy of information recommendation.
By obtaining the user preference time series, a pre-trained preference prediction model is determined, which includes a vector transformation layer, an encoder, a decoder, and a prediction layer, and the decoder includes a mask attention mechanism layer and a non-mask attention mechanism layer. The model generates user preference prediction information through the interaction of the encoder, mask attention mechanism layer and non-mask attention mechanism layer.
Through the interaction and entanglement between the encoder and the decoder, user interest characteristics are accurately extracted, improving the accuracy of information recommendation.
Smart Images
Figure CN119598020A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an information recommendation method, device and equipment. Background Art
[0002] Sequential recommendation mainly mines time-aware user interests by simulating sequential behaviors. However, the existing transformerer has shortcomings and lacks interaction and entanglement between encoders and decoders, which makes it impossible to accurately extract user interest features, further leading to inaccurate information recommendations. Summary of the invention
[0003] In view of this, the present invention provides an information recommendation method, device and equipment, which can solve the technical problem that the user interest features cannot be accurately extracted, which further leads to inaccurate information recommendation.
[0004] According to a first aspect of the present invention, there is provided an information recommendation method, the method comprising:
[0005] Obtaining a user preference time series and determining a pre-trained preference prediction model, wherein the preference prediction model includes a vector conversion layer, an encoder, a decoder, and a prediction layer, and the decoder includes a masked attention mechanism layer and a non-masked attention mechanism layer;
[0006] The user preference time series is input into the preference prediction model, the user preference time series is converted into an embedding vector through the vector conversion layer, the encoding vector corresponding to the embedding vector is determined through the encoder, the intermediate decoding vector corresponding to the encoding vector is determined through the masked attention mechanism layer, the coefficient matrix of the non-masked attention mechanism layer is determined according to the encoding vector, and then the decoding vector corresponding to the intermediate decoding vector is determined through the non-masked attention mechanism layer, and the preference prediction information corresponding to the decoding vector is determined through the prediction layer;
[0007] Based on the preference prediction information, user recommendation information is determined.
[0008] Preferably, determining, by the encoder, the encoding vector corresponding to the embedded vector comprises:
[0009] For each layer of the encoding network of the encoder, the current layer query matrix, current layer key-value matrix and current layer value matrix corresponding to the previous layer encoding vector are calculated through the encoding network, and the current layer encoding vector is determined according to the current layer query matrix, the current layer key-value matrix and the current layer value matrix, wherein the previous layer encoding vector corresponding to the first layer of the encoding network is the embedding vector, and the current layer encoding vector corresponding to the last layer of the encoding network is the encoding vector corresponding to the encoder.
[0010] Preferably, after determining the coefficient matrix of the non-masked attention mechanism layer according to the encoding vector and before determining the decoding vector corresponding to the intermediate decoding vector through the non-masked attention mechanism layer, the method further includes:
[0011] A query matrix of a non-masked attention mechanism layer is calculated according to the encoding vector, and the query matrix is determined as a coefficient matrix of the non-masked attention mechanism layer.
[0012] Preferably, determining the coefficient matrix of the non-masked attention mechanism layer according to the encoding vector and then determining the decoding vector corresponding to the intermediate decoding vector through the non-masked attention mechanism layer includes:
[0013] Calculate the key-value matrix and the value matrix of the non-masked attention mechanism layer according to the intermediate decoded vector;
[0014] The decoding vector corresponding to the intermediate decoding vector is calculated according to the coefficient matrix, the key-value matrix and the value matrix through the non-masked attention mechanism layer.
[0015] Preferably, before determining the pre-trained preference prediction model, the method further comprises:
[0016] Acquire a historical user preference time series for training, and input the historical user preference time series into a transformer model to be trained, wherein the transformer model to be trained includes a vector conversion layer to be trained, an encoder to be trained, a decoder to be trained, and a prediction layer to be trained;
[0017] Inputting the historical user preference time series into the transformer model to be trained, and converting the historical user preference time series into an embedding vector to be trained through the vector conversion layer to be trained;
[0018] Determine the encoding vector to be trained corresponding to the embedding vector to be trained by the encoder to be trained, determine the intermediate decoding vector to be trained corresponding to the encoding vector to be trained by the masked attention mechanism layer to be trained, determine the coefficient matrix to be trained of the non-masked attention mechanism layer to be trained according to the encoding vector to be trained, and then determine the decoding vector to be trained corresponding to the intermediate decoding vector to be trained by the non-masked attention mechanism layer to be trained;
[0019] The loss function is calculated, and the parameters of the transformer model to be trained are adjusted until the loss function converges to obtain a pre-trained preference prediction model.
[0020] Preferably, the calculation of the loss function includes:
[0021] Calculate the encoder attention loss function according to the label vector corresponding to the encoder to be trained and the encoding vector to be trained;
[0022] Calculate a decoder attention loss function based on the label vector corresponding to the decoder to be trained and the decoding vector to be trained;
[0023] Obtaining preset sub-weights corresponding to the encoder attention loss function and the decoder attention loss function, respectively, and calculating the attention loss function according to the encoder attention loss function, the decoder attention loss function and the preset sub-weights;
[0024] Calculating a codec loss function according to the decoding vector to be trained and the historical embedding vector;
[0025] Obtain preset weights corresponding to the codec loss function and the attention loss function, and calculate the loss function according to the preset weights, the codec loss function, and the attention loss function.
[0026] Preferably, converting the user preference time series into an embedding vector through the vector conversion layer includes:
[0027] Acquire user preference information and user time information from the user preference time series;
[0028] Through the vector conversion layer, the user preference information is converted into a preference vector, the user time information is converted into a time vector, and the preference vector and the time vector are fused to obtain an embedding vector corresponding to the user preference time series.
[0029] According to a second aspect of the present invention, there is provided an information recommendation device, the device comprising:
[0030] An acquisition module, used to acquire a user preference time series and determine a pre-trained preference prediction model, wherein the preference prediction model includes a vector conversion layer, an encoder, a decoder and a prediction layer, and the decoder includes a masked attention mechanism layer and a non-masked attention mechanism layer;
[0031] A coding and decoding module, used for inputting the user preference time series into the preference prediction model, converting the user preference time series into an embedding vector through the vector conversion layer, determining the encoding vector corresponding to the embedding vector through the encoder, determining the intermediate decoding vector corresponding to the encoding vector through the masked attention mechanism layer, determining the coefficient matrix of the non-masked attention mechanism layer according to the encoding vector, determining the decoding vector corresponding to the intermediate decoding vector through the non-masked attention mechanism layer, and determining the preference prediction information corresponding to the decoding vector through the prediction layer;
[0032] A determination module is used to determine user recommendation information based on the preference prediction information.
[0033] According to a third aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned information recommendation method is implemented.
[0034] According to a fourth aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned information recommendation method when executing the program.
[0035] By means of the above technical scheme, the present invention provides an information recommendation method, device and equipment. The present invention can first obtain a user preference time series and determine a pre-trained preference prediction model, wherein the preference prediction model includes a vector conversion layer, an encoder, a decoder and a prediction layer, and the decoder includes a masked attention mechanism layer and a non-masked attention mechanism layer; then, the user preference time series is input into the preference prediction model, and the user preference time series is converted into an embedding vector through the vector conversion layer, and further, the encoding vector corresponding to the embedding vector is determined through the encoder, and the intermediate decoding vector corresponding to the encoding vector is determined through the masked attention mechanism layer, and the coefficient matrix of the non-masked attention mechanism layer is determined according to the encoding vector, and then the decoding vector corresponding to the intermediate decoding vector is determined through the non-masked attention mechanism layer, and the preference prediction information corresponding to the decoding vector is determined through the prediction layer; finally, based on the preference prediction information, the user recommendation information is determined. Through the technical solution of the present invention, since the coefficient matrix of the non-masked attention mechanism layer is determined by the encoding vector output by the encoder, instead of the prior art in which the coefficient matrix of the non-masked attention mechanism layer is determined by the intermediate decoding vector output by the decoder, the interaction and entanglement between the encoder and the decoder is realized, the user interest features can be accurately extracted, and further the information can be accurately recommended.
[0036] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation on the local application. In the drawings:
[0038] Figure 1A schematic diagram showing a flow chart of an information recommendation method provided by an embodiment of the present invention;
[0039] Figure 2 An internal structure diagram of a preference prediction model provided by an embodiment of the present invention is shown;
[0040] Figure 3 A schematic diagram showing the structure of an information recommendation device provided by an embodiment of the present invention is shown;
[0041] Figure 4 A schematic diagram showing the structure of another information recommendation device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0042] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.
[0043] This embodiment provides an information recommendation method, such as Figure 1 and Figure 2 As shown, the method includes:
[0044] 101. Obtain a user preference time series and determine a pre-trained preference prediction model, wherein the preference prediction model includes a vector conversion layer, an encoder, a decoder, and a prediction layer, and the decoder includes a masked attention mechanism layer and a non-masked attention mechanism layer.
[0045] Here, U represents the user set U = {u1,u2,…,u m}, m represents the number of users; V represents the preference set V = {v1,v2,…,v n}, n represents the number of preferences; t represents the time step; then is the preference set V of the user set U at time step t, which is arranged in chronological order (e.g. ascending order) to form the user preference time series T represents the maximum length of the sequence. The goal is to use the pre-trained preference prediction model to predict the user's preference at a time step greater than T based on the user's preference time series.
[0046] For this embodiment, Figure 2 As shown in FIG. 1 , the internal structure of the preference prediction model is a vector conversion layer connected to the encoder, the encoder is connected to the masked attention mechanism layer of the decoder, the masked attention mechanism layer of the decoder is connected to the unmasked attention mechanism layer of the decoder, and the unmasked attention mechanism layer of the decoder is connected to the prediction layer. The input to the preference prediction model is the user preference time series, which first passes through the vector conversion layer and finally the prediction layer outputs the preference prediction information, that is, the preference prediction model outputs the preference prediction information.
[0047] Preferably, the following takes the case where the pre-trained preference prediction model is a pre-trained transformer model as an example to explain how to train the transformer model to be trained to obtain the pre-trained transformer model.
[0048] Before determining the pre-trained preference prediction model, it also includes: obtaining the historical user preference time series for training, inputting the historical user preference time series into the transformer model to be trained, wherein the transformer model to be trained includes a vector conversion layer to be trained, an encoder to be trained, a decoder to be trained and a prediction layer to be trained; inputting the historical user preference time series into the transformer model to be trained, and converting the historical user preference time series into an embedding vector to be trained through the vector conversion layer to be trained; determining the encoding vector to be trained corresponding to the embedding vector to be trained through the encoder to be trained, determining the intermediate decoding vector to be trained corresponding to the encoding vector to be trained through the masked attention mechanism layer to be trained, and determining the training coefficient matrix of the non-masked attention mechanism layer to be trained according to the encoding vector to be trained, and then determining the decoding vector to be trained corresponding to the intermediate decoding vector to be trained through the non-masked attention mechanism layer to be trained; calculating the loss function, adjusting the parameters of the transformer model to be trained until the loss function converges, and obtaining the pre-trained preference prediction model.
[0049] It should be noted that as long as the attention mechanism is involved, these three different weight matrices will be used, and the training process is the same as the prediction process in step 102 below.
[0050] The difference is that during the training process, the weight matrix is to be trained, while the prediction process in step 102 uses the trained weight matrix.
[0051] The difference is on the other hand: during the training process, what is input to the encoder to be trained is the embedding vector to be trained and the label vector corresponding to the encoder to be trained, while in the prediction process of step 102, only the embedding vector is input to the encoder, because the label vector corresponding to the encoder to be trained is the correct vector, which is used to calculate the encoder attention loss function with the encoding vector to be trained output by the encoder to be trained. Similarly, what is input to the decoder to be trained is the decoding vector to be trained and the label vector corresponding to the decoder to be trained, while in the prediction process of step 102, only the decoding vector is input to the decoder.
[0052] Calculate the loss function, including:
[0053] Calculate the attention loss function: calculate the encoder attention loss function according to the label vector corresponding to the encoder to be trained and the encoding vector to be trained; calculate the decoder attention loss function according to the label vector corresponding to the decoder to be trained and the decoding vector to be trained; obtain the preset sub-weights corresponding to the encoder attention loss function and the decoder attention loss function, and calculate the attention loss function according to the encoder attention loss function, the decoder attention loss function and the preset sub-weights;
[0054] Among them, the label vector is the correct vector fed in. It should be noted that the attention loss function corresponds to the query weight matrix, the key weight matrix, and the value weight matrix. On the one hand, as long as the attention mechanism is involved, these three different weight matrices will be used. On the other hand, the convergence of the attention loss function represents the completion of the training of the three different weight matrices.
[0055] Calculate the codec loss function: Calculate the codec loss function based on the decoding vector to be trained and the historical embedding vector;
[0056] The preset weights corresponding to the codec loss function and the attention loss function are obtained, and the loss function is calculated according to the preset weights, the codec loss function and the attention loss function.
[0057] 102. Input the user preference time series into the preference prediction model, convert the user preference time series into an embedding vector through the vector conversion layer, determine the encoding vector corresponding to the embedding vector through the encoder, determine the intermediate decoding vector corresponding to the encoding vector through the masked attention mechanism layer, determine the coefficient matrix of the non-masked attention mechanism layer according to the encoding vector, then determine the decoding vector corresponding to the intermediate decoding vector through the non-masked attention mechanism layer, and determine the preference prediction information corresponding to the decoding vector through the prediction layer.
[0058] First, regarding "converting the user preference time series into an embedded vector through the vector conversion layer", since the transformer model uses the attention mechanism to extract information, the attention mechanism itself cannot obtain position information, that is, context. Therefore, in order to make the embedded vector input into the encoder have position information.
[0059] Specifically: obtain user preference information and user time information from the user preference time series; through the vector conversion layer, convert the user preference information into a preference vector, convert the user time information into a time vector, and fuse the preference vector and the time vector to obtain the embedding vector corresponding to the user preference time series. The specific implementation method of converting the user preference information into a preference vector and converting the user time information into a time vector can refer to the existing technology, for example, through network training or through some trained models to convert it into a continuous vector, which is not limited here.
[0060] Second, as for “determining the encoding vector corresponding to the embedding vector by the encoder”, as an implementation method, Figure 2 As shown, for each layer of the encoding network of the encoder, the current layer query matrix, current layer key-value matrix and current layer value matrix corresponding to the previous layer encoding vector are calculated through the encoding network, and the current layer encoding vector is determined according to the current layer query matrix, current layer key-value matrix and current layer value matrix, wherein the previous layer encoding vector corresponding to the first layer encoding network is the embedding vector, and the current layer encoding vector corresponding to the last layer encoding network is the encoding vector corresponding to the encoder.
[0061] It should be noted that each layer of the encoding network of the encoder uses a multi-head attention mechanism for calculation. Specifically, the embedded vector is input to the encoder as the input of the first layer of the encoding network, and the output of the first layer of the encoding network is calculated using the multi-head attention mechanism as an example. The same is true for the remaining layers of the encoding network, so it will not be repeated. Get three different weight matrices corresponding to a head of the first layer, namely the first layer query weight matrix, the first layer key value weight matrix, and the first layer value weight matrix (these three different weight matrices are all obtained through pre-training), and then multiply them with the embedding vector respectively to obtain the first layer query matrix, the first layer key value matrix, and the first layer value matrix corresponding to this head. Calculate the transpose multiplication of the first layer query matrix and the first layer key value matrix to obtain the attention score matrix corresponding to this head. Use the Softmax function to normalize the attention score matrix so that the sum of the attention scores of each row of the attention score matrix is 1. Multiply the normalized attention score matrix with the first layer value matrix to obtain the first layer sub-encoding vector corresponding to this head. All heads have the same calculation process. Concatenate the first layer sub-encoding vectors obtained from all heads to obtain the first layer encoding vector, until the last layer encoding network outputs the last layer encoding vector as the encoding vector corresponding to the encoder.
[0062] It should be noted that the query weight matrix of each layer is different, and the query weight matrices of the same layer and different heads are also different. The same is true for the key weight matrix and the value weight matrix, which will not be repeated here.
[0063] Among them, the Softmax function can compress a K-dimensional vector z containing any real number into another K-dimensional real vector σ(z), so that the range of each element is between (0, 1).
[0064] Among them, for the query matrix, key-value matrix, and value matrix, the value matrix is a value sequence. Each value in the sequence is initially considered to be independent of each other and has limited information. If you want each value to contain more information, one way is to let each value fuse the information of other values in the sequence. In the fusion method, if the averaging method is used, a large amount of irrelevant information will be integrated. A better way is to weight the sum according to the correlation between the value and other values. So how to calculate the correlation between each value in the value matrix and other values, and quantify the correlation into a weight value for the final weighted summation? At this time, the query matrix and the key-value matrix are used. The query matrix asks each value in the key-value matrix which value is most relevant to me. After comparing each value in the key-value matrix with the query matrix, a similarity with the query matrix is obtained to form a similarity sequence. The similarity sequence is normalized into a sequence with a sum of 1, which can be regarded as a weight. This weight is the weight quantification of the correlation between the corresponding value in the value matrix and other values. In short, the query matrix queries the key-value matrix to provide a weight for the weighted summation between each value in the value matrix and other values.
[0065] It should be noted that the essence of the attention mechanism is a set of weight coefficients learned autonomously by the network, and a mechanism that emphasizes the areas of interest while suppressing irrelevant background areas in a "dynamically weighted" manner. When predicting, this dynamic weighting indicates which parts of the input sequence to focus on. That is, the essence of the attention mechanism comes from the human visual attention mechanism. When people look at things, they generally don't look at everything from beginning to end. They often only observe and pay attention to a specific part according to their needs. It is a weight parameter allocation mechanism, and the goal is to assist the model in capturing important information. The multi-head attention mechanism corresponds to multiple subspaces. Compared with the single-head attention mechanism, it can focus on information in different subspaces, thereby extracting information more accurately.
[0066] Third, for "determining the coefficient matrix of the non-masked attention mechanism layer according to the encoding vector and then determining the decoding vector corresponding to the intermediate decoding vector through the non-masked attention mechanism layer", first, before this, the query matrix of the non-masked attention mechanism layer is calculated according to the encoding vector, and the query matrix is determined as the coefficient matrix of the non-masked attention mechanism layer. Then, the key-value matrix and the value matrix of the non-masked attention mechanism layer are calculated according to the intermediate decoding vector; through the non-masked attention mechanism layer, the decoding vector corresponding to the intermediate decoding vector is calculated according to the coefficient matrix, the key-value matrix and the value matrix.
[0067] Among them, Figure 2 As shown, the decoder includes at least one layer of decoding network, and each layer of the decoding network includes a non-masked attention mechanism layer and a masked attention mechanism layer (masking refers to covering the value after the current value and predicting the current value based on the value before the current value).
[0068] Taking the first-layer decoding network as an example, first, the decoder is input with the decoding vector output by the encoder. The decoding vector is first input to the masked attention mechanism layer in the first-layer decoding network (the masked attention mechanism layer also uses the multi-head attention mechanism). The calculation process performed in the masked attention mechanism layer is the same as that in the first-layer encoding network, which will not be repeated here. The intermediate decoding vector of the first layer output by the masked attention mechanism layer is obtained, and then the intermediate decoding vector of the first layer is input to the non-masked attention mechanism layer in the first-layer decoding network.
[0069] In the non-masked attention mechanism layer, taking one head in the multi-head attention mechanism as an example, three different query weight matrices, key-value weight matrices, and value weight matrices of one head of the non-masked attention mechanism layer are obtained. The calculation of the key-value matrix and the value matrix are the same as in the encoding network, that is, the intermediate decoding vector of the first layer is multiplied with the key-value weight matrix and the value weight matrix of the non-masked attention mechanism layer to obtain the key-value matrix and the value matrix, while the calculation of the query matrix is different from that of the encoding network. The difference is that, in the non-masked attention mechanism layer, the encoding vector corresponding to the encoder is multiplied with the query weight matrix of the non-masked attention mechanism layer, rather than the intermediate decoding vector of the first layer output by the masked attention mechanism layer, thereby realizing the interaction and entanglement between the encoder and the decoder, which can further accurately extract user interest features and improve the accuracy of information recommendation.
[0070] Further, the transpose multiplication of the query matrix and the key-value matrix is calculated to obtain the attention score matrix corresponding to this head. The attention score matrix is normalized using the Softmax function so that the sum of the attention scores of each row of the attention score matrix is 1. The normalized attention score matrix is multiplied by the value matrix to obtain the first-layer sub-decoding vector corresponding to this head. All heads have the same calculation process. The first-layer sub-decoding vectors obtained from all heads are concatenated to obtain the first-layer decoding vector. The first-layer decoding vector is input into the masked attention mechanism layer of the second-layer decoding network to obtain the intermediate decoding vector of the second layer. The intermediate decoding vector of the second layer is input into the non-masked attention mechanism layer of the second-layer decoding network. The intermediate decoding vector of the second layer is multiplied with the key-value weight matrix and value weight matrix of the non-masked attention mechanism layer to obtain the key-value matrix and value matrix. Similarly, the encoding vector corresponding to the encoder is multiplied with the query weight matrix of the non-masked attention mechanism layer. The rest will not be repeated until the non-masked attention mechanism layer of the last layer of the decoding network outputs the decoding vector corresponding to the decoder.
[0071] By inputting the decoding vector into the prediction layer, the preference prediction information can be output. As an implementation method, this preference prediction information can be the preference probability of all users in the user preference time series for the preferred objects in the user preference time series compared to at least one future moment in the user preference time series.
[0072] 103. Determine user recommendation information based on the preference prediction information.
[0073] As an implementation method, taking the example that the preference prediction information is the preference probability of all users in the user preference time series for the preferred objects in the user preference time series at least one future moment compared to the user preference time series, the preferred objects whose preference probabilities are ranked first by a preset number can be recommended to the user.
[0074] The present invention provides an information recommendation method, device and equipment. The present invention can first obtain a user preference time series and determine a pre-trained preference prediction model, wherein the preference prediction model includes a vector conversion layer, an encoder, a decoder and a prediction layer, and the decoder includes a masked attention mechanism layer and a non-masked attention mechanism layer; then, the user preference time series is input into the preference prediction model, and the user preference time series is converted into an embedding vector through the vector conversion layer, and further, the encoding vector corresponding to the embedding vector is determined through the encoder, and the intermediate decoding vector corresponding to the encoding vector is determined through the masked attention mechanism layer, and the coefficient matrix of the non-masked attention mechanism layer is determined according to the encoding vector. Then, the decoding vector corresponding to the intermediate decoding vector is determined through the non-masked attention mechanism layer, and the preference prediction information corresponding to the decoding vector is determined through the prediction layer; finally, based on the preference prediction information, the user recommendation information is determined. Through the technical solution of the present invention, since the coefficient matrix of the non-masked attention mechanism layer is determined by the encoding vector output by the encoder, instead of the prior art in which the coefficient matrix of the non-masked attention mechanism layer is determined by the intermediate decoding vector output by the decoder, the interaction and entanglement between the encoder and the decoder is realized, the user interest features can be accurately extracted, and further the information can be accurately recommended.
[0075] Further, as Figure 1 The specific implementation of the method shown in the figure, the embodiment of the present invention provides an information recommendation device, such as Figure 3 As shown, the device includes: an acquisition module 31, a coding and decoding module 32, and a determination module 33;
[0076] An acquisition module 31 is used to acquire a user preference time series and determine a pre-trained preference prediction model, wherein the preference prediction model includes a vector conversion layer, an encoder, a decoder and a prediction layer, and the decoder includes a masked attention mechanism layer and a non-masked attention mechanism layer;
[0077] The encoding and decoding module 32 is used to input the user preference time series into the preference prediction model, convert the user preference time series into an embedding vector through the vector conversion layer, determine the encoding vector corresponding to the embedding vector through the encoder, determine the intermediate decoding vector corresponding to the encoding vector through the masked attention mechanism layer, determine the coefficient matrix of the non-masked attention mechanism layer according to the encoding vector, determine the decoding vector corresponding to the intermediate decoding vector through the non-masked attention mechanism layer, and determine the preference prediction information corresponding to the decoding vector through the prediction layer;
[0078] The determination module 33 is used to determine user recommendation information based on the preference prediction information.
[0079] Correspondingly, in order to determine the encoding vector corresponding to the embedded vector through the encoder, the encoding and decoding module 32 can be specifically used to calculate the query matrix, key-value matrix and value matrix of the previous layer corresponding to the encoding vector of the previous layer for each layer of the encoding network of the encoder through the encoding network, and determine the encoding vector of the current layer according to the query matrix, key-value matrix and value matrix of the current layer, wherein the encoding vector of the previous layer corresponding to the first layer of the encoding network is the embedding vector, and the encoding vector of the current layer corresponding to the last layer of the encoding network is the encoding vector corresponding to the encoder.
[0080] In specific application scenarios, such as Figure 4 As shown, an information recommendation device also includes: a coefficient matrix determination module 34, which can be specifically used to calculate the query matrix of the non-masked attention mechanism layer according to the encoding vector, and determine the query matrix as the coefficient matrix of the non-masked attention mechanism layer.
[0081] Correspondingly, in order to determine the coefficient matrix of the non-masked attention mechanism layer according to the encoding vector and then determine the decoding vector corresponding to the intermediate decoding vector through the non-masked attention mechanism layer, the encoding and decoding module 32 can also be used to calculate the key-value matrix and the value matrix of the non-masked attention mechanism layer according to the intermediate decoding vector; through the non-masked attention mechanism layer, the decoding vector corresponding to the intermediate decoding vector is calculated according to the coefficient matrix, the key-value matrix and the value matrix.
[0082] In specific application scenarios, such as Figure 4As shown, an information recommendation device, the device also includes: a training module 35, which can be specifically used to obtain a historical user preference time series for training, and input the historical user preference time series into a transformer model to be trained, wherein the transformer model to be trained includes a vector conversion layer to be trained, an encoder to be trained, a decoder to be trained and a prediction layer to be trained; the historical user preference time series is input into the transformer model to be trained, and the historical user preference time series is converted into an embedding vector to be trained through the vector conversion layer to be trained; the encoding vector to be trained corresponding to the embedding vector to be trained is determined through the encoder to be trained, the intermediate decoding vector to be trained corresponding to the encoding vector to be trained is determined through the masked attention mechanism layer to be trained, and the training coefficient matrix of the non-masked attention mechanism layer to be trained is determined according to the encoding vector to be trained, and then the decoding vector to be trained corresponding to the intermediate decoding vector to be trained is determined through the non-masked attention mechanism layer to be trained; the loss function is calculated, and the parameters of the transformer model to be trained are adjusted until the loss function converges to obtain a pre-trained preference prediction model.
[0083] Correspondingly, in order to calculate the loss function, the training module 35 can also be specifically used to calculate the encoder attention loss function according to the label vector corresponding to the encoder to be trained and the encoding vector to be trained; calculate the decoder attention loss function according to the label vector corresponding to the decoder to be trained and the decoding vector to be trained; obtain the preset sub-weights corresponding to the encoder attention loss function and the decoder attention loss function, and calculate the attention loss function according to the encoder attention loss function, the decoder attention loss function and the preset sub-weights; calculate the codec loss function according to the decoding vector to be trained and the historical embedding vector; obtain the preset weights corresponding to the codec loss function and the attention loss function, and calculate the loss function according to the preset weights, the codec loss function and the attention loss function.
[0084] Correspondingly, in order to convert the user preference time series into an embedded vector through the vector conversion layer, the encoding and decoding module 32 can also be used to obtain user preference information and user time information from the user preference time series; through the vector conversion layer, the user preference information is converted into a preference vector, the user time information is converted into a time vector, and the preference vector and the time vector are fused to obtain the embedded vector corresponding to the user preference time series.
[0085] It should be noted that for other corresponding descriptions of the functional units involved in the information recommendation device provided in this embodiment, reference can be made to Figure 1 The corresponding description will not be repeated here.
[0086] Based on the above Figure 1 The method shown in the embodiment, accordingly, also provides a storage medium, which can be volatile or non-volatile, and stores a computer program, which is executed by a processor to implement the above-mentioned Figure 1 The information shown is recommended method.
[0087] Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of the present invention.
[0088] Based on the above Figure 1 The method shown and Figure 3 , Figure 4 In order to achieve the above-mentioned purpose, the present embodiment further provides a computer device, which includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 The information shown is recommended method.
[0089] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.
[0090] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.
[0091] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned computer device and supports the operation of information processing programs and other software and / or programs. The network communication module is used to communicate between the components inside the storage medium and to communicate with other hardware and software in the information processing entity device.
[0092] Through the description of the above implementation modes, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform, or by hardware.
[0093] The present invention provides an information recommendation method, device and equipment. The present invention can first obtain a user preference time series and determine a pre-trained preference prediction model, wherein the preference prediction model includes a vector conversion layer, an encoder, a decoder and a prediction layer, and the decoder includes a masked attention mechanism layer and a non-masked attention mechanism layer; then, the user preference time series is input into the preference prediction model, and the user preference time series is converted into an embedding vector through the vector conversion layer, and further, the encoding vector corresponding to the embedding vector is determined through the encoder, and the intermediate decoding vector corresponding to the encoding vector is determined through the masked attention mechanism layer, and the coefficient matrix of the non-masked attention mechanism layer is determined according to the encoding vector. Then, the decoding vector corresponding to the intermediate decoding vector is determined through the non-masked attention mechanism layer, and the preference prediction information corresponding to the decoding vector is determined through the prediction layer; finally, based on the preference prediction information, the user recommendation information is determined. Through the technical solution of the present invention, since the coefficient matrix of the non-masked attention mechanism layer is determined by the encoding vector output by the encoder, instead of the prior art in which the coefficient matrix of the non-masked attention mechanism layer is determined by the intermediate decoding vector output by the decoder, the interaction and entanglement between the encoder and the decoder is realized, the user interest features can be accurately extracted, and further the information can be accurately recommended.
[0094] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present invention. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.
[0095] The above serial numbers of the present invention are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of the present invention, but the present invention is not limited thereto, and any changes that can be thought of by those skilled in the art should fall within the protection scope of the present invention.
Claims
1. An information recommendation method, characterized in that: The method comprises: Obtaining a user preference time series and determining a pre-trained preference prediction model, wherein the preference prediction model includes a vector conversion layer, an encoder, a decoder, and a prediction layer, and the decoder includes a masked attention mechanism layer and a non-masked attention mechanism layer; The user preference time series is input into the preference prediction model, the user preference time series is converted into an embedding vector through the vector conversion layer, the encoding vector corresponding to the embedding vector is determined through the encoder, the intermediate decoding vector corresponding to the encoding vector is determined through the masked attention mechanism layer, the coefficient matrix of the non-masked attention mechanism layer is determined according to the encoding vector, and then the decoding vector corresponding to the intermediate decoding vector is determined through the non-masked attention mechanism layer, and the preference prediction information corresponding to the decoding vector is determined through the prediction layer; Based on the preference prediction information, user recommendation information is determined.
2. The method according to claim 1, characterized in that The determining, by the encoder, the encoding vector corresponding to the embedded vector comprises: For each layer of the encoding network of the encoder, the current layer query matrix, current layer key-value matrix and current layer value matrix corresponding to the previous layer encoding vector are calculated through the encoding network, and the current layer encoding vector is determined according to the current layer query matrix, the current layer key-value matrix and the current layer value matrix, wherein the previous layer encoding vector corresponding to the first layer of the encoding network is the embedding vector, and the current layer encoding vector corresponding to the last layer of the encoding network is the encoding vector corresponding to the encoder.
3. The method according to claim 1, characterized in that After determining the coefficient matrix of the non-masked attention mechanism layer according to the encoding vector and before determining the decoding vector corresponding to the intermediate decoding vector through the non-masked attention mechanism layer, the method further includes: A query matrix of a non-masked attention mechanism layer is calculated according to the encoding vector, and the query matrix is determined as a coefficient matrix of the non-masked attention mechanism layer.
4. The method according to claim 1 or 3, characterized in that: The step of determining the coefficient matrix of the non-masked attention mechanism layer according to the encoding vector and then determining the decoding vector corresponding to the intermediate decoding vector through the non-masked attention mechanism layer includes: Calculate the key-value matrix and the value matrix of the non-masked attention mechanism layer according to the intermediate decoded vector; The decoding vector corresponding to the intermediate decoding vector is calculated according to the coefficient matrix, the key-value matrix and the value matrix through the non-masked attention mechanism layer.
5. The method according to claim 1, characterized in that: Before determining the pre-trained preference prediction model, the method further includes: Acquire a historical user preference time series for training, and input the historical user preference time series into a transformer model to be trained, wherein the transformer model to be trained includes a vector conversion layer to be trained, an encoder to be trained, a decoder to be trained, and a prediction layer to be trained; Inputting the historical user preference time series into the transformer model to be trained, and converting the historical user preference time series into an embedding vector to be trained through the vector conversion layer to be trained; Determine the encoding vector to be trained corresponding to the embedding vector to be trained by the encoder to be trained, determine the intermediate decoding vector to be trained corresponding to the encoding vector to be trained by the masked attention mechanism layer to be trained, determine the coefficient matrix to be trained of the non-masked attention mechanism layer to be trained according to the encoding vector to be trained, and then determine the decoding vector to be trained corresponding to the intermediate decoding vector to be trained by the non-masked attention mechanism layer to be trained; The loss function is calculated, and the parameters of the transformer model to be trained are adjusted until the loss function converges to obtain a pre-trained preference prediction model.
6. The method according to claim 5, characterized in that The calculation loss function includes: Calculate the encoder attention loss function according to the label vector corresponding to the encoder to be trained and the encoding vector to be trained; Calculate a decoder attention loss function based on the label vector corresponding to the decoder to be trained and the decoding vector to be trained; Obtaining preset sub-weights corresponding to the encoder attention loss function and the decoder attention loss function, respectively, and calculating the attention loss function according to the encoder attention loss function, the decoder attention loss function and the preset sub-weights; Calculating a codec loss function according to the decoding vector to be trained and the historical embedding vector; Obtain preset weights corresponding to the codec loss function and the attention loss function, and calculate the loss function according to the preset weights, the codec loss function, and the attention loss function.
7. The method according to claim 1, characterized in that The converting the user preference time series into an embedding vector through the vector conversion layer includes: Acquire user preference information and user time information from the user preference time series; Through the vector conversion layer, the user preference information is converted into a preference vector, the user time information is converted into a time vector, and the preference vector and the time vector are fused to obtain an embedding vector corresponding to the user preference time series.
8. An information recommendation device, characterized in that: The device comprises: An acquisition module, used to acquire a user preference time series and determine a pre-trained preference prediction model, wherein the preference prediction model includes a vector conversion layer, an encoder, a decoder and a prediction layer, and the decoder includes a masked attention mechanism layer and a non-masked attention mechanism layer; A coding and decoding module, used for inputting the user preference time series into the preference prediction model, converting the user preference time series into an embedding vector through the vector conversion layer, determining the encoding vector corresponding to the embedding vector through the encoder, determining the intermediate decoding vector corresponding to the encoding vector through the masked attention mechanism layer, determining the coefficient matrix of the non-masked attention mechanism layer according to the encoding vector, determining the decoding vector corresponding to the intermediate decoding vector through the non-masked attention mechanism layer, and determining the preference prediction information corresponding to the decoding vector through the prediction layer; A determination module is used to determine user recommendation information based on the preference prediction information.
9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the information recommendation method according to any one of claims 1 to 7 is implemented.
10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the information recommendation method according to any one of claims 1 to 7 is implemented.