A cross-domain sequence intelligent recommendation method based on time-gated recurrent units
By using time-gated loop units in the recommendation system to model the cross-domain user behavior sequences and integrate time intervals and user information, the data sparse and cold start problems are solved, and the accuracy and personalized effect of the recommendation system are improved, so as to achieve The effectiveness of cross-domain recommendations.
Patent Information
- Application Number
- CN202111313875.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-08
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-11-08
AI Technical Summary
Existing recommendation systems face sparse data and cold start problems, especially in cross-field recommendations, which fail to effectively utilize time intervals and user information, resulting in poor recommendation results.
Using a cross-domain sequence intelligent recommendation method based on time-gated loop units, the user's historical behavior sequences in the source and target fields are merged, time intervals and user information are integrated, and the improved GRU model is used to model it, and through Two domain-specific subnets predict candidate behavior probability distributions.
It improves the accuracy and efficiency of cross-domain recommendations, can effectively recommend in user cold start scenarios, enhances personalized recommendation effects, and captures the mutual influence of behaviors between the two fields, improving recommendation performance.
Smart Images

Figure CN114036380B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of information retrieval and recommendation system, and specifically designs a cross-domain sequence intelligent recommendation method based on time-gated cyclic units. Background Art
[0002] With the booming development of the Internet industry, information overload is becoming more and more obvious, and users have limited energy and it is difficult to obtain content that they are interested in. The birth of the recommendation system is to solve the above problems, and the recommendation system also faces many challenges, such as data sparsity and cold start. Some current recommendation systems, whether the number of users or the number of items, are often in the hundreds of millions, and each user usually does not interact with many items, which is the problem of data sparsity. The cold start problem can be divided into: user cold start, item cold start and system cold start. The problem to be solved in user cold start is how to make recommendations to new users; item cold start is to study how to recommend new items to users; for a new system, there is no user historical behavior information, and making recommendations in this case is a system cold start.
[0003] At present, there have been a lot of studies on data sparsity and cold start problems, and cross-domain recommendation is one of them. Cross-domain recommendation can combine data from multiple fields, improve the recommendation effect through methods such as transfer learning and feature interaction, and alleviate data sparsity and some cold start problems to a certain extent. Collaborative filtering is a relatively classic method in the recommendation field. It uses the feature vectors of users and items to score users and items; this method is simple to implement, but does not consider user historical behavior information. The so-called sequence modeling is to model the user's historical behavior sequence and recommend items that the user may be interested in; commonly used methods include recurrent neural networks, attention mechanisms, and Transformers. Compared with traditional methods, sequence recommendation has better results. Cross-domain sequence recommendation (CDSR) is to model the user's historical behavior sequence in multiple fields at the same time, and then recommend multiple fields. At present, most CDSR methods use recurrent neural networks to model sequences, but these methods all assume that the time interval between each two adjacent items in the sequence is consistent. In addition, in terms of personalized recommendations, most sequence modeling methods do not introduce user information into the gate mechanism when modeling.
[0004] After searching, the closest prior art 1, CN109544306B, a cross-domain recommendation method based on user behavior sequence features, including: obtaining the target user's interaction project sequence in the auxiliary domain and the target domain respectively, obtaining the target user's user embedding matrix and project embedding matrix in the auxiliary domain and the target domain respectively through the embedding layer of the cross-domain recommendation model, and then obtaining the target user's total preference features in the auxiliary domain and the target domain respectively through the LSTM layer, and finally, after the MLP layer processing, the user's total preference features in the auxiliary domain are transferred to the target domain, and the target user's preference score for each project in the target domain is obtained by calculation, and the target user is recommended projects that may be purchased or visited in the next period of time in the target domain. The application of the technical solution provided by the present invention improves the recommendation accuracy of the target domain and improves the recommendation performance.
[0005] The closest prior art 2, CN113297487A, a sequence recommendation system based on attention mechanism enhanced gated recurrent unit, characterized in that it includes a data preprocessing module, an input sequence embedding module, an information extraction module, an information enhancement module, and a recommendation module; the data preprocessing module is used to construct the input sample of the model, the input sample is the user's behavior sequence information, and the module determines the sequence of the user's behavior sequence according to the order of occurrence time; the input sequence embedding module is used to convert the unique hot variable into a dense vector, and the unique hot variable is the output of the data preprocessing module; the information extraction module extracts from the embedding vector The user preference information is obtained, and the preference information includes long-term preference and short-term preference information and is obtained by a gated recurrent unit structure. The gated recurrent unit outputs the current preference information at each time moment, uses the last time moment information as the short-term preference information, and uses the arithmetic mean of the preference as the long-term preference information; the information enhancement module enhances the output information of the information extraction module to capture more information about the user preference; the recommendation module matches the output of the information enhancement module with the candidate recommendation items on the test set, and evaluates the recommendation performance of the model, and at the same time sorts them according to the matching scores, obtains several items with the highest scores, and recommends them to the user.
[0006] The method proposed in Patent 1, "A cross-domain recommendation method and device based on user behavior sequence characteristics", is to model the historical interaction project sequences of the source domain and the target domain respectively, and the sequence modeling uses LSTM, while we model the cross-domain sequence after the source domain and the target domain are merged, and the sequence modeling uses the time-gated recurrent unit proposed by us. Our method takes into account time intervals and personalization issues. In addition, Patent 1 is only for improving the recommendation effect of the target domain, while our method is for improving the recommendation effect of both domains at the same time.
[0007] Patent 2 "A Sequence Recommendation System and Method Based on Attention Mechanism Enhanced Gated Recurrent Unit" models the historical behavior sequence of a single domain. The sequence modeling adopts GRU. Our method is to model the cross-domain sequence, and the modeling method is the GRU improved by us, which takes the time interval and user information into consideration. Summary of the invention
[0008] The present invention aims to solve the problems of data sparsity, item cold start, time interval, personalization, etc. in the above prior art, and proposes a cross-domain sequence intelligent recommendation method based on time-gated recurrent unit. The technical solution of the present invention is as follows:
[0009] A cross-domain sequence intelligent recommendation method based on time-gated recurrent units comprises the following steps:
[0010] Obtain data sets from two domains, where the two domains refer to the source domain and the target domain, and perform preprocessing on the data sets from the two domains, including user and item filtering; merge the user's historical behavior sequences in the two domains into a cross-domain sequence in chronological order; use time-gated recurrent units to model the cross-domain sequence, where the time-gated recurrent unit refers to an improvement on the GRU that incorporates time intervals and user information; obtain the probability distribution of candidate behaviors in the two domains through modeling of two domain-specific sub-networks, where the two domain-specific sub-networks are the source domain-specific sub-network and the target domain-specific sub-network, and the network structure is composed of a feedforward neural network, and then make the next behavior recommendation for the two domains at the same time.
[0011] Furthermore, the data preprocessing of the data sets of the two fields includes: counting the overlapping users U=U in the source field and the target field α ∩U β , U α Represents the user set in the source domain, U β Represents the user set of the target domain; then filter out the behavior records of non-overlapping users in the source domain to obtain Item Collection S, and then filter out the behavior records of non-overlapping users in the target domain to obtain Item Collection T .
[0012] Furthermore, the specific steps of merging the user's historical behavior sequences in the two fields into one sequence in chronological order are as follows: for each user u∈U, merge its behavior sequences in the two fields into one sequence in chronological order in, k represents the total number of historical behaviors of the user in the two fields.
[0013] Furthermore, the time-gated recurrent unit is mainly composed of three parts: a reset gate ri , update gate z i and candidate activation status The reset gate and candidate activation state are consistent with GRU, and the update gate is composed of the user gate u i and time gate i It is obtained through matrix transformation, where the user gate determines the user's preference for the current item, and the time gate reflects the influence of the current item on the next item; finally, the feature vector h of the last time step of DTGRU is taken out t-1 As a holistic characterization of the behavior of both domains.
[0014] Further, update gate z i By user portal t , Time Gate t Through matrix transformation, we get:
[0015]
[0016]
[0017] z i =σ(W τ u i +W υ g i )
[0018] Among them, φ represents the tanh activation function, σ represents the sigmoid activation function, and W u , V u , W τ , W υ represents the weight matrix, b z , b t represents the bias term, Δt i Represents the sequence s u The time interval between the i-th item and the i+1-th item in the equation. In addition, the time interval Δt i The type is different, the parameters The value of is also different:
[0019]
[0020] There are 4 different types of time intervals:
[0021]
[0022] Among them, domain i Indicates which field the i-th item belongs to, source indicates the source field, and target indicates the target field.
[0023] Furthermore, the candidate activation state
[0024]
[0025] Among them, φ t represents the tanh activation function, W n , V n represents the weight matrix, represents the embedding vector corresponding to the i-th behavior of user u, r i Represents the reset gate at the i-th behavior.
[0026] Furthermore, the two domain-specific sub-networks are: source domain-specific sub-network f S (x) and the target domain specific sub-network f T (x);
[0027]
[0028]
[0029] Among them, γ represents the relu activation function, represents the weight matrix, Represents the bias term.
[0030] Furthermore, the probability distribution of all candidate items in the two domains is obtained through the domain-specific subnetwork, specifically including:
[0031]
[0032]
[0033] After obtaining the probability distribution of the two fields, the items with the highest probability in each field can be recommended to the user at the same time. The model loss function is The cross entropy loss of the two fields is added, and then the stochastic gradient descent algorithm is used to optimize the model parameters:
[0034]
[0035]
[0036]
[0037] Among them, λ is an adjustable parameter used to control the proportion of the regularization term, and Θ represents the set of all model parameters.
[0038] The advantages and beneficial effects of the present invention are as follows:
[0039] 1. This method models two domains at the same time and predicts both domains at the same time. It is more efficient than the single-domain sequence modeling method and alleviates the data sparsity problem to a certain extent. In addition, the two domains can promote each other in the interaction process and improve the recommendation effect of the two domains at the same time. First of all, the existing data is often presented in the form of a single domain, such as movie and music data, so most of the existing sequence modeling methods are based on a single domain; while user behavior is influenced by multiple domains. Although there are many cross-domain methods now, many methods use the source domain to assist the target domain in learning and only improve the target domain effect. Even if there are methods that consider improving both domains at the same time, they are not based on sequence modeling.
[0040] 2. This method can be applied to user cold start scenarios. For example, if a user only has historical behavior records in the source domain but not in the target domain, we can still use the sequence in a single domain as our input and make predictions and recommendations for both domains at the same time, which is not available in single-domain recommendation methods.
[0041] 3. This method designs a time gate that considers the time interval between two adjacent historical behaviors, and differentiates the model parameters according to the different fields to which the two behaviors belong, expressing the influence of different types of time intervals in a more fine-grained manner, which plays a better role in recommendation. In addition, we also design a user gate that introduces user preference information to play a role in personalized recommendation.
[0042] In terms of structure, the two comparison papers use native LSTM and GRU respectively, and both are based on single-domain sequence modeling. Our method models the fused sequence and can capture the mutual influence of behaviors between different domains. In addition, since the native GRU does not take into account the time interval and user interests, we have improved the native GRU by adding time gates and user gates to capture the connection between adjacent items and user preferences. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a flowchart of a cross-domain sequence intelligent recommendation method based on a time-gated cyclic unit according to a preferred embodiment of the present invention;
[0044] Figure 2 is a schematic diagram of a cross-domain sequence recommendation model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0045] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.
[0046] The technical solution of the present invention to solve the above technical problems is:
[0047] like Figure 1 As shown, a cross-domain sequence intelligent recommendation method based on time-gated recurrent units includes the following steps:
[0048] Step 101 , preprocessing the data set, merging the user's historical behavior sequences in two fields into one sequence in chronological order.
[0049] Specifically, the preprocessing of the data set includes: firstly, counting the overlapping users U=U in the source domain and the target domain S ∩U T , then filter out the behavior records that do not belong to overlapping users in the source domain to obtain the item set S, and then filter out the behavior records that do not belong to overlapping users in the target domain to obtain the item set T.
[0050] Specifically, merging the user's historical behavior sequences in two fields into one sequence in chronological order means: for each user u∈U, merging its behavior sequences in two fields into one sequence in chronological order in, k represents the total number of historical behaviors of the user in the two fields. u Each item in The next interactive item The time interval between them is denoted as Δt i , considering the fields to which the current item and the next item belong, Δt i There are 4 types:
[0051]
[0052] Let the sequence length of the input model be L. We need to model the previous L historical behaviors of user u to predict its next behavior in two areas. The input of the model includes a sequence of item embedding vectors User embedding vector U and time interval vector
[0053] Step 102 , using a time-gated recurrent unit to model the cross-domain sequence.
[0054] See also Figure 2 As shown, the calculation formula of the time gated recurrent unit (DTGRU) is:
[0055]
[0056]
[0057]
[0058] z i =σ(W τ u i +W υ g i )
[0059]
[0060]
[0061] Among them, φ represents the tanh activation function, σ represents the sigmoid activation function, and W r , V r , W u , V u , W τ , W υ W n , V n represents the weight matrix, b r , b z , b t , b n represents the bias term, Δt i Represents the sequence s u The time interval between the i-th item and the i+1-th item in . In addition, the time interval Δt of each time step i The type is different, the parameters The value of is also different:
[0062]
[0063] DTGRU is mainly composed of three parts: reset gate r i , update gate z i and candidate activation status Among them, the reset gate and candidate activation state are consistent with GRU, and the update gate is composed of the user gate u i and time gate i It is obtained through matrix transformation. Among them, the user gate determines the user's preference for the current item, and the time gate reflects the influence of the current item on the next item. Finally, take out the feature vector h of the last time step of DTGRU t-1 As a holistic characterization of the behavior of both domains.
[0064] Step 103, the probability distribution of candidate behaviors in the two domains is obtained respectively by modeling two domain-specific sub-networks, and then the next behavior is recommended for the two domains at the same time.
[0065] Specifically, we define two domain-specific subnetworks f S , f T :
[0066]
[0067]
[0068] Among them, γ represents the relu activation function, represents the weight matrix, Represents the bias term. The probability distribution of all candidate items in the two fields is obtained through the domain-specific subnetwork:
[0069]
[0070]
[0071] After obtaining the probability distribution of the two fields, the items with the highest probability in each field can be recommended to the user at the same time. Model loss function The cross entropy loss of the two fields is added, and then the stochastic gradient descent algorithm is used to optimize the model parameters:
[0072]
[0073]
[0074]
[0075] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0076] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.
Claims
1. A cross-domain sequence intelligent recommendation method based on time-gated recurrent units, It is characterized in that The following steps are involved: Obtain data sets from two domains, the two domains refer to the source domain and the target domain, and perform preprocessing on the data sets from the two domains, including user and item filtering; merge the user's historical behavior sequences in the two domains into a cross-domain sequence in chronological order; use a time-gated recurrent unit to model the cross-domain sequence, which is an improvement on the GRU that incorporates time intervals and user information; obtain the probability distribution of candidate behaviors in the two domains through modeling of two domain-specific subnetworks, the two domain-specific subnetworks are the source domain-specific subnetwork and the target domain-specific subnetwork, and the network structure is composed of a feedforward neural network, and then make the next behavior recommendation for the two domains at the same time; The time gated recurrent unit DTGRU refers to: i , update gate z i and candidate activation status The reset gate and candidate activation state are consistent with GRU, and the update gate is composed of the user gate u i and time gate i It is obtained through matrix transformation, where the user gate determines the user's preference for the current item, and the time gate reflects the influence of the current item on the next item; finally, the feature vector h of the last time step of DTGRU is taken out t-1 as an overall characterization of the behavior of both domains; Update gate z i By user portal t , Time Gate t Through matrix transformation, we get: With i =σ(W τ at i +W v g i ) Among them, φ represents the tanh activation function, σ represents the sigmoid activation function, and W u , V u , W τ , W v represents the weight matrix, b z , b t represents the bias term, Δt i Represents the sequence s u The time interval between the i-th item and the i+1-th item in the equation. In addition, the time interval Δt i The type is different, the parameters The value of is also different: There are 4 different types of time intervals: Among them, domain i Indicates which domain the i-th item belongs to, source indicates the source domain, and target indicates the target domain; Candidate activation status: Among them, φ t represents the tanh activation function, W n , V n represents the weight matrix, represents the embedding vector corresponding to the i-th behavior of user u, r i represents the reset gate at the i-th behavior; The two domain-specific sub-networks are: source domain-specific sub-network f S (x) and the target domain specific sub-network f T (x); Among them, γ represents the relu activation function, represents the weight matrix, represents the bias term; The probability distribution of all candidate items in the two domains is obtained through the domain-specific sub-network, specifically including: After obtaining the probability distribution of the two fields, the items with the highest probability in each field can be recommended to the user at the same time. The model loss function is The cross entropy loss of the two fields is added, and then the stochastic gradient descent algorithm is used to optimize the model parameters: Among them, λ is an adjustable parameter used to control the proportion of the regularization term, and Θ represents the set of all model parameters.
2. According to claim 1, a cross-domain sequence intelligent recommendation method based on time-gated recurrent unit, It is characterized in that The data preprocessing of the data sets of the two fields includes: counting the overlapping users U=U in the source field and the target field α ∩U β , U α Represents the user set in the source domain, U β Represents the user set of the target domain; then filter out the behavior records of non-overlapping users in the source domain to obtain Item Collection S, and then filter out the behavior records that do not belong to overlapping users in the target field to obtain the item set T.
3. According to claim 2, a cross-domain sequence intelligent recommendation method based on time-gated recurrent units, It is characterized in that The specific steps of merging the user's historical behavior sequences in two fields into one sequence in chronological order are as follows: for each user u∈U, merge its behavior sequences in two fields into one sequence in chronological order in, k represents the total number of historical behaviors of the user in the two fields.
