Multi-service recommendation method, device and equipment

Abstract

The invention provides a multi-service recommendation method, device and equipment, and the method comprises the steps: inputting user feature data into a multi-service recommendation model, and obtaining the purchase confidence of a user for each service or product output by the multi-service recommendation model; wherein the multi-service recommendation model utilizes a sparse attention mechanism to optimize a feature matrix of the user. According to the embodiment of the invention, the standard attention mechanism in the multi-service recommendation model is optimized, and the sparse attention mechanism is used for learning, so that the influence of feature redundancy and invalid information on the multi-service recommendation model caused by association of unassociated features is avoided; and through the convex combination of the feature representation matrix in the multi-service recommendation model, potential association between different columns can be mined by fully utilizing the structural characteristics of the sparse matrix, and meanwhile, information redundancy and interference caused by unnecessary association are avoided, so that the model performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a multi-service recommendation method, device and equipment. Background Art

[0002] With the rapid development of mobile communications and the Internet, the market demand for services and products such as mobile traffic, bandwidth, and rights is increasing, and functions are gradually iteratively improved. In order to better complete the precision marketing tasks of mobile services and products and improve the business conversion rate, it is necessary to quickly predict and accurately locate high-value users with potential needs in the market. In precision marketing tasks, a common scenario is to quickly determine the mobile services and products that users are most likely to purchase and apply for from a large number of alternative mobile services and products when users actively call in and inquire about mobile products, so as to achieve multi-service recommendations for users. Given alternative mobile services and products, user multi-service precision recommendation belongs to a multi-classification problem. Through user feature data, a user multi-service precision recommendation model is constructed, and the confidence of users purchasing and applying for alternative services and products is output. According to the size of the confidence, the mobile services and products that users are most likely to purchase and apply for are screened out.

[0003] The solution for multi-service recommendation in the prior art mainly forms annotated data through user feature data and the behavior of handling mobile services or purchasing mobile products in the future to build and train multi-service recommendation models. User feature data mainly includes static features such as user age and gender and statistical features of user online behavior. The data composed of these features belongs to tabular data, that is, each feature in the data has actual business meaning, and there is a lack of obvious spatial and contextual associations between different features. At present, modeling based on tabular data is mainly based on tree models such as xgboost (an optimized distributed gradient boosting library). These tree models have high performance and stability in tabular data. With the development of deep neural network technology, deep neural networks based on hybrid structures and transformers are also widely used in multi-service precision recommendation tasks. In addition, some business-based criteria can be used to build user multi-service recommendation models in specific scenarios, and by conducting market surveys on specific mobile products and user groups, user product purchase criteria with a certain scope of application are formed.

[0004] However, user feature data belongs to tabular data. When the user feature dimension is high, the tree model cannot effectively represent the continuous and discrete features of the data, so it has limitations in model performance. Although the modeling method based on deep neural network can build a representation model for data features, it cannot integrate the structural characteristics of tabular data, so it cannot effectively extract the potential associations between different fields of tabular data, thus limiting the model performance. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a multi-business recommendation method, device and equipment to solve the problem that the modeling method based on deep neural network in the prior art cannot integrate the structural characteristics of tabular data, and therefore cannot effectively extract the potential correlation between different fields of tabular data, thereby limiting the model performance.

[0006] In order to solve the above problem, an embodiment of the present invention provides a multi-service recommendation method, including:

[0007] Inputting the user feature data into the multi-service recommendation model, and obtaining the user's purchase confidence for each service or product output by the multi-service recommendation model;

[0008] The multi-service recommendation model optimizes the user's feature matrix using a sparse attention mechanism.

[0009] Wherein, the multi-service recommendation model includes:

[0010] An input processor, used to take user features as input and output a feature matrix for each user;

[0011] The neural network module based on the attention mechanism is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the vector of the corresponding row of the optimized feature representation matrix;

[0012] And, a classifier module is used to take the optimized feature representation matrix as input and output the purchase probability of the user for each business or product.

[0013] Wherein, the input processor is further used for:

[0014] Construct an overall feature matrix based on the user features in the data set;

[0015] A feature matrix for each user feature is determined according to the type of the user feature and the overall feature matrix.

[0016] Wherein, the neural network module based on the attention mechanism includes: a multi-head attention mechanism module and a feedforward neural network FFN block module;

[0017] The multi-head attention mechanism module is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the optimized feature representation matrix;

[0018] The FFN block module is used to transform and optimize the row vectors of the feature representation matrix output by the multi-head attention mechanism module through a fully connected network to obtain the vectors of the corresponding rows of the optimized feature representation matrix.

[0019] Wherein, the method further comprises:

[0020] Configure the first trainable parameter matrix W Q , the second trainable parameter matrix W K , the third trainable parameter matrix W V and a sparse matrix M;

[0021] Construct the vector CDEAtt(PX i ,W Q ,W K ,W V ,M); where X i represents the user's feature matrix; W Q represents the first trainable parameter matrix; W K represents the second trainable parameter matrix; W V represents the training parameter matrix of the third lesson; M represents a sparse matrix; P represents X i as input and output R n×n The output of P satisfies the following conditions:

[0022] P kj ≥0,k,j=1,…,n; and

[0023] Wherein, the sparse matrix M has the following form:

[0024]

[0025] Among them, M1, M2 and M3 are pre-defined block diagonal sparse matrices respectively; M1 corresponds to the first type of variables, and the size of M1 is the same as the number of the first type of variables; M2 corresponds to the second type of continuous variables, and the size of M2 is the same as the number of the second type of variables; M3 corresponds to the third type of variables, and the size of M3 is the same as the number of the third type of variables; -inf represents an infinitesimal quantity.

[0026] Among them, the neural network P has the following form:

[0027]

[0028] Among them, P1, P2 and P3 are matrices respectively; P1 corresponds to the first type of variables, and the size of P1 is the same as the number of the first type of variables; P2 corresponds to the second type of variables, and the size of P2 is the same as the number of the second type of variables; P3 corresponds to the third type of variables, and the size of P3 is the same as the number of the third type of variables.

[0029] Wherein, the multi-head attention mechanism module is further used for:

[0030] Based on the vector of the sparse attention mechanism, the calculation formula for determining the multi-head attention mechanism module is:

[0031] MultiHeadCDEAtt=Concat(CDEAtt 1 ,…,CDEAtt C )W o ;

[0032] Among them, Concat means that the vectors are concatenated according to the last dimension; CDEAtt 1 ,…,CDEAtt C Represents the vector of each sparse attention mechanism; W o Represents the preset matrix corresponding to the multi-head attention mechanism;

[0033]

[0034] Wherein, the classifier module is further used for:

[0035] The concatenated vector of the corresponding row of the optimized feature representation matrix output by the multi-head attention mechanism module is used as input, and the output vector O with a length of C is i ;

[0036] Among them, O i The kth component of indicates whether the user purchases the kth service or product.

[0037] Wherein, the method further comprises:

[0038] Adopting an adversarial contrast learning strategy to conduct adversarial training on the multi-service recommendation model to obtain the multi-service recommendation model; the adversarial contrast learning strategy includes: a target task for multi-task recommendation and a data forgery identification learning task;

[0039] The target task of the multi-task recommendation is used to predict the purchase confidence of each business or product based on real user characteristics;

[0040] The data forgery identification learning task is used to train a multi-service recommendation model to identify real user features by forging user features.

[0041] Wherein, the method further comprises:

[0042] Construct fake user features based on random column concatenation of different user features;

[0043] By training models based on real user features and the forged user features, a binary classification training task of a multi-service recommendation model is constructed; the binary classification training task includes: a target task for multi-task recommendation and a data forgery identification learning task;

[0044] Among them, the objective function of model training is:

[0045]

[0046] Among them, CDEAttNN represents the multi-service recommendation model; D L Represents a set of real user features; D f represents a set of forged user features; the objective function performs end-to-end training on all learnable parameters in CDEAttNN.

[0047] The embodiment of the present invention further provides a multi-service recommendation device, including:

[0048] An acquisition module, used to input user feature data into a multi-service recommendation model, and obtain the user's purchase confidence for each service or product output by the multi-service recommendation model;

[0049] The multi-service recommendation model optimizes the user's feature matrix using a sparse attention mechanism.

[0050] An embodiment of the present invention further provides a multi-service recommendation device, including a processor and a transceiver, wherein the transceiver receives and sends data under the control of the processor, and the processor is used to perform the following operations:

[0051] Inputting the user feature data into the multi-service recommendation model, and obtaining the user's purchase confidence for each service or product output by the multi-service recommendation model;

[0052] The multi-service recommendation model optimizes the user's feature matrix using a sparse attention mechanism.

[0053] Wherein, the multi-service recommendation model includes:

[0054] An input processor, used to take user features as input and output a feature matrix for each user;

[0055] The neural network module based on the attention mechanism is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the vector of the corresponding row of the optimized feature representation matrix;

[0056] And, a classifier module is used to take the optimized feature representation matrix as input and output the purchase probability of the user for each business or product.

[0057] Wherein, the input processor is further used for:

[0058] Construct an overall feature matrix based on the user features in the data set;

[0059] A feature matrix for each user feature is determined according to the type of the user feature and the overall feature matrix.

[0060] Wherein, the neural network module based on the attention mechanism includes: a multi-head attention mechanism module and a feedforward neural network FFN block module;

[0061] The multi-head attention mechanism module is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the optimized feature representation matrix;

[0062] The FFN block module is used to transform and optimize the row vectors of the feature representation matrix output by the multi-head attention mechanism module through a fully connected network to obtain the vectors of the corresponding rows of the optimized feature representation matrix.

[0063] The processor is further configured to perform the following operations:

[0064] Configure the first trainable parameter matrix W Q , the second trainable parameter matrix W K , the third trainable parameter matrix W V and a sparse matrix M;

[0065] Construct the vector CDEAtt(PX i ,W Q ,W K ,W V ,M); where X i represents the user's feature matrix; W Q represents the first trainable parameter matrix; W K represents the second trainable parameter matrix; W V represents the training parameter matrix of the third lesson; M represents a sparse matrix; P represents X i as input and output R n×n The output of P satisfies the following conditions:

[0066] P kj ≥0,k,j=1,…,n; and

[0067] Wherein, the sparse matrix M has the following form:

[0068]

[0069] Among them, M1, M2 and M3 are pre-defined block diagonal sparse matrices respectively; M1 corresponds to the first type of variables, and the size of M1 is the same as the number of the first type of variables; M2 corresponds to the second type of continuous variables, and the size of M2 is the same as the number of the second type of variables; M3 corresponds to the third type of variables, and the size of M3 is the same as the number of the third type of variables; -inf represents an infinitesimal quantity.

[0070] Among them, the neural network P has the following form:

[0071]

[0072] Among them, P1, P2 and P3 are matrices respectively; P1 corresponds to the first type of variables, and the size of P1 is the same as the number of the first type of variables; P2 corresponds to the second type of variables, and the size of P2 is the same as the number of the second type of variables; P3 corresponds to the third type of variables, and the size of P3 is the same as the number of the third type of variables.

[0073] Wherein, the multi-head attention mechanism module is further used for:

[0074] Based on the vector of the sparse attention mechanism, the calculation formula for determining the multi-head attention mechanism module is:

[0075] MultiHeadCDEAtt=Concat(CDEAtt 1 ,…,CDEAtt C )W o ;

[0076] Among them, Concat means that the vectors are concatenated according to the last dimension; CDEAtt 1 ,…,CDEAtt C Represents the vector of each sparse attention mechanism; W o Represents the preset matrix corresponding to the multi-head attention mechanism;

[0077]

[0078] Wherein, the classifier module is further used for:

[0079] The concatenated vector of the corresponding row of the optimized feature representation matrix output by the multi-head attention mechanism module is used as input, and the output vector O with a length of C is i ;

[0080] Among them, O i The kth component of indicates whether the user purchases the kth service or product.

[0081] The processor is further configured to perform the following operations:

[0082] Adopting an adversarial contrast learning strategy to conduct adversarial training on the multi-service recommendation model to obtain the multi-service recommendation model; the adversarial contrast learning strategy includes: a target task for multi-task recommendation and a data forgery identification learning task;

[0083] The target task of the multi-task recommendation is used to predict the purchase confidence of each business or product based on real user characteristics;

[0084] The data forgery identification learning task is used to train a multi-service recommendation model to identify real user features by forging user features.

[0085] The processor is further configured to perform the following operations:

[0086] Construct fake user features based on random column concatenation of different user features;

[0087] By training models based on real user features and the forged user features, a binary classification training task of a multi-service recommendation model is constructed; the binary classification training task includes: a target task for multi-task recommendation and a data forgery identification learning task;

[0088] Among them, the objective function of model training is:

[0089]

[0090] Among them, CDEAttNN represents the multi-service recommendation model; D L Represents a set of real user features; D f represents a set of forged user features; the objective function performs end-to-end training on all learnable parameters in CDEAttNN.

[0091] An embodiment of the present invention further provides a multi-service recommendation device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor implements the multi-service recommendation method as described above when executing the program.

[0092] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the multi-service recommendation method described above are implemented.

[0093] The above technical solution of the present invention has at least the following beneficial effects:

[0094] In the multi-service recommendation method, device and equipment of the embodiments of the present invention, the standard attention mechanism in the multi-service recommendation model is optimized, and the sparse attention mechanism learning is used to avoid the influence of feature redundancy and invalid information caused by associating unrelated features on the multi-service recommendation model; and through the convex combination of the feature representation matrix in the multi-service recommendation model of the present application, the structural characteristics of the sparse matrix can be fully utilized to explore the potential associations between different columns, while avoiding information redundancy and interference caused by unnecessary associations, thereby improving model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 A schematic diagram showing the steps of a multi-service recommendation method provided by an embodiment of the present invention;

[0096] Figure 2 A schematic diagram showing the structure of a multi-service recommendation model in a multi-service recommendation method provided in an embodiment of the present invention;

[0097] Figure 3 A schematic diagram showing calculation of an attention matrix in a multi-service recommendation model provided by an embodiment of the present invention;

[0098] Figure 4 A framework diagram showing data forgery identification training in a multi-service recommendation model training process provided by an embodiment of the present invention;

[0099] Figure 5 A schematic diagram showing the structure of a multi-service recommendation device provided by an embodiment of the present invention;

[0100] Figure 6 A schematic diagram showing the structure of a multi-service recommendation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0101] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0102] like Figure 1 As shown, an embodiment of the present invention provides a multi-service recommendation method, including:

[0103] Step 101, inputting user feature data into a multi-service recommendation model, and obtaining the user's purchase confidence for each service or product output by the multi-service recommendation model;

[0104] The multi-service recommendation model optimizes the user's feature matrix using a sparse attention mechanism.

[0105] The embodiment of the present invention improves the traditional attention mechanism for user feature data in multi-service recommendation scenarios, and designs a multi-service recommendation model including a new attention mechanism CDEAtt. This model can solve the problem that user feature data is tabular data and lacks spatial association and structural association, resulting in the predefined sparse matrix destroying the potential association information between different columns of the tabular data, which leads to the degradation of model performance. Through the convex combination of the feature representation matrix in CDEAtt, the structural characteristics of the sparse matrix can be fully utilized to mine the potential association between different columns, while avoiding information redundancy and interference caused by unnecessary associations, thereby improving model performance.

[0106] In at least one embodiment of the present invention, Figure 2 As shown, the multi-service recommendation model includes:

[0107] An input processor, used to take user features as input and output a feature matrix for each user;

[0108] The attention-based neural network (Transformer) module is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the vector of the corresponding row of the optimized feature representation matrix;

[0109] And, a classifier module is used to take the optimized feature representation matrix as input and output the purchase probability of the user for each business or product.

[0110] Due to the lack of significant spatial and contextual associations between different fields of tabular data in the prior art, the multi-business recommendation model based on deep neural networks lacks an effective module for mining the associations between different fields of tabular data, resulting in the performance and stability of the multi-business recommendation model being limited. The embodiment of the present invention optimizes the standard Transformer neural network structure and utilizes the existing sparse self-attention (masked self-attention) learning to avoid the impact of feature redundancy and invalid information caused by associating unrelated features on the multi-business recommendation model. In addition, based on the structural characteristics of tabular data, an adaptive learning model for the association of different fields is constructed on the basis of the sparse attention mechanism, thereby improving the performance of the traditional Transformer network structure.

[0111] Assume that the basic situation of basic symbols and user characteristic data is as follows:

[0112] Assume that there are C businesses or products that need to be recommended. Given a data set Where N represents the number of samples, x i ∈R n represents the user feature vector, y i ∈{0,1}C Indicates whether the user will conduct business or purchase products in the future. i The i-th component of is 1, indicating that the user purchased the i-th product; otherwise, y i The i-th component of is 0, indicating that the user did not purchase the i-th product. The target task of the multi-service recommendation model CDEAttNN is to L Typical prediction tasks. In the embodiment of the present invention, the multi-service recommendation model CDEAttNN adopts a multi-task learning mode to perform joint training under the target task and data forgery identification learning strategy. The field names of the user feature data for multi-service recommendation are shown in Table 2:

[0113] Table 2, Field table of user feature data

[0114]

[0115]

[0116] All feature data of users need to be divided into three categories. The first type of continuous features belong to the dynamic features of the user for the month, which have a certain randomness as time goes by (can be referred to as the first type of variables). The second type of features belong to static features, which have a certain trend as time goes by (can be referred to as the second type of variables). The third type of continuous features belong to traffic features, which mainly characterize the traffic behavior of the user for the month (can be referred to as the third type of variables).

[0117] As an optional embodiment, the input processor is further configured to:

[0118] Construct an overall feature matrix based on the user features in the data set;

[0119] A feature matrix for each user feature is determined according to the type of the user feature and the overall feature matrix.

[0120] For example, the input processor converts the user features As input, output each user's feature matrix X i First, for the dataset D L The user feature components in construct the overall feature matrix. i When the jth component of is a continuous feature, the vector e is defined as j ∈R m When all user feature data x i When the jth component of is a discrete feature, the matrix is ​​defined as Where m represents x i The dimension of the feature vector corresponding to each component in is n jRepresents the number of values ​​of discrete features. Based on the overall feature matrix, each user feature x can be defined i The feature matrix X i ∈R n×m When the jth feature is a continuous feature, X i The jth row of is defined as x j e j When the jth feature is a discrete feature and its value is e j When the kth row of i The jth row of the matrix e is defined as j The kth row of .

[0121] As another optional embodiment, Figure 2 As shown, the attention mechanism-based neural network (Transformer) module includes: a multi-head attention mechanism (MultiHeadCDEAtt) module and a feedforward neural network FFN block (FFN block) module;

[0122] The multi-head attention mechanism module is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the optimized feature representation matrix;

[0123] The FFN block module is used to transform and optimize the row vectors of the feature representation matrix output by the multi-head attention mechanism module through a fully connected network to obtain the vectors of the corresponding rows of the optimized feature representation matrix.

[0124] In the embodiment of the present invention, the Transformer module mainly includes a MultiHeadCDEAtt module and a FFN block module. In the MultiHeadCDEAtt module, the present invention mainly improves the data structure of the traditional attention mechanism for user features and proposes a new attention mechanism variant. The MultiHeadCDEAtt module mainly performs a i The feature representation matrix X i Optimize and output the optimized representation matrix X i 1 , whose matrix dimensions and size are the same as X i In the FFN block module, the fully connected network is used to transform and optimize the representation of each row of the feature matrix.

[0125] In at least one embodiment of the present invention, the method further comprises:

[0126] Configure the first trainable parameter matrix W Q , the second trainable parameter matrix W K, the third trainable parameter matrix W V and a sparse matrix M;

[0127] Construct the vector CDEAtt(PX i ,W Q ,W K ,W V ,M); where X i represents the user's feature matrix; W Q represents the first trainable parameter matrix; W K represents the second trainable parameter matrix; W V represents the training parameter matrix of the third lesson; M represents a sparse matrix; P represents X i as input and output R n×n The output of P satisfies the following conditions:

[0128] P kj ≥0,k,j=1,…,n; and

[0129] Optionally, W Q ∈R m×m ; W K ∈R m×m ; W V ∈R m×m ; M∈{0,-∞} n×n ; The existing sparse attention mechanism is defined as follows:

[0130] Maskatt(X i ,W Q ,W k ,W v M):=AX i W V ;

[0131] The attention matrix is ​​defined as follows:

[0132]

[0133] Where d∈R. The sparse matrix M can effectively control X i The features of are involved in the calculation of the attention matrix A. For the features of multi-service recommendation data, the embodiment of the present invention constructs a matrix M of a block diagonal matrix type according to different feature categories, so that features of the same category can transfer information. The sparse matrix M has the following form:

[0134]

[0135] Among them, M1, M2 and M3 are pre-defined block diagonal sparse matrices respectively; M1 corresponds to the first type of variables, and the size of M1 is the same as the number of the first type of variables; M2 corresponds to the second type of continuous variables, and the size of M2 is the same as the number of the second type of variables; M3 corresponds to the third type of variables, and the size of M3 is the same as the number of the third type of variables; -inf represents an infinitesimal quantity.

[0136] Based on the sparse attention Maskatt mechanism, the embodiment of the present invention defines a new attention mechanism CDEAtt (X i ,W Q ,W K ,W V ,M,P), which has the following form:

[0137] CDEAtt(PX i ,W Q ,W K ,W V ,M)(1);

[0138] Where P is X i as input, and the output is R n×n The neural network, and the output satisfies the following conditions:

[0139] P kj ≥0,k,j=1,…,n;

[0140] Combined with the classification of user feature data, the neural network P has the following form:

[0141]

[0142] Among them, P1, P2 and P3 are matrices respectively; P1 corresponds to the first type of variables, and the size of P1 is the same as the number of the first type of variables; P2 corresponds to the second type of variables, and the size of P2 is the same as the number of the second type of variables; P3 corresponds to the third type of variables, and the size of P3 is the same as the number of the third type of variables.

[0143] Optionally, in the new attention mechanism CDEAtt for tabular data defined in the embodiment of the present invention (also referred to as the sparse attention mechanism), the attention matrix is ​​calculated as follows: Figure 3 Based on the optimized CDEAtt attention mechanism, the multi-head attention mechanism module is further used to:

[0144] Based on the vector of the sparse attention mechanism, the calculation formula for determining the multi-head attention mechanism module is:

[0145] MultiHeadCDEAtt=Concat(CDEAtt 1 ,…,CDEAttC )W o ;

[0146] Among them, Concat means that the vectors are concatenated according to the last dimension; CDEAtt 1 ,…,CDEAtt C Represents the vector of each sparse attention mechanism; W o Represents the preset matrix corresponding to the multi-head attention mechanism; represents the trainable parameter matrix corresponding to the j-th attention mechanism, j=1...C.

[0147] As an optional embodiment, the FFN block module is a fully connected network, and the input is the optimized feature representation matrix The row vector of The vector corresponding to the row, where The dimensions and sizes of same.

[0148] As an optional embodiment, the classifier module is further used for:

[0149] The concatenated vector of the corresponding row of the optimized feature representation matrix output by the multi-head attention mechanism module is used as input, and the output vector O with a length of C is i ;

[0150] Among them, O i The kth component of indicates whether the user purchases the kth service or product.

[0151] Among them, the classifier module is a fully connected network, and the input is the feature representation matrix in the FFN block module The vector after concatenation of each row Output vector O of length C i ; Among them O i The kth component of indicates whether the user purchases the kth product or service.

[0152] In the prior art, the multi-service recommendation model based on deep neural network has a large number of parameters to be learned in the attention mechanism model representing data features, and the existing model parameter training method mainly trains these parameters based on the similarity represented by the tabular data features. However, the compatibility of the predefined similarity and the classification module in the multi-service recommendation model cannot be guaranteed, so the training of these model parameters cannot effectively improve the classification performance of the classification model, and is therefore limited in predicting the accuracy of user purchases of products. In response to the above problems, the embodiment of the present invention proposes a learning strategy based on data forgery identification. By constructing perturbation data of user features, the multi-service recommendation model is trained to identify real data and perturbation data. Through this representation learning task, the multi-service recommendation model can more accurately characterize the characteristics of the user data distribution, thereby reducing the model performance incompatibility problem caused by the incompatibility of the representation learning task and the model classification task, thereby effectively improving the model performance of the multi-service recommendation.

[0153] As an optional embodiment, the method further includes:

[0154] Adopting an adversarial contrast learning strategy to conduct adversarial training on the multi-service recommendation model to obtain the multi-service recommendation model; the adversarial contrast learning strategy includes: a target task for multi-task recommendation and a data forgery identification learning task;

[0155] The target task of the multi-task recommendation is used to predict the purchase confidence of each business or product based on real user characteristics;

[0156] The data forgery identification learning task is used to train a multi-service recommendation model to identify real user features by forging user features.

[0157] In other words, the process of training the multi-service recommendation model mainly includes the target task of multi-service recommendation and the data forgery identification learning task. The target task of multi-service recommendation is to predict and output the purchase confidence of each service given the user feature data. The data forgery identification task aims to construct some forged user feature data to allow the multi-service recommendation model to further identify which are user feature data and which are not user feature data, and finally allow the multi-service recommendation model to better characterize the user data distribution characteristics and improve the prediction effect of the multi-service recommendation model. The main framework of data forgery identification training is as follows: Figure 4 shown.

[0158] In the process of training the model, in the data forgery identification training, the method further includes:

[0159] Construct fake user features based on random column concatenation of different user features;

[0160] By training models based on real user features and the forged user features, a binary classification training task of a multi-service recommendation model is constructed; the binary classification training task includes: a target task for multi-task recommendation and a data forgery identification learning task;

[0161] Among them, the objective function of model training is:

[0162]

[0163] Among them, CDEAttNN represents the multi-service recommendation model; D L Represents a set of real user features; D f represents a set of forged user features; the objective function performs end-to-end training on all learnable parameters in CDEAttNN.

[0164] For example, construct fake user feature data. The fake feature data is mainly generated based on the random column splicing of different user feature data. Assume that the number of the first category variables is n1, the number of the second category variables is n2, and the number of the third category variables is n3. For the given hyperparameter γ∈(0,1), randomly extract n1γ dimensions from the first category of features, randomly extract n2γ dimensions from the second category of features, and randomly extract n3γ dimensions from the third category of features from the n-dimensional user data. The sum of these dimensions is recorded as index1, and the unextracted user feature dimensions are recorded as index2. For the given user feature data x i and x j , then the synthesized fake user feature data x1 ′ The k-th dimension component of is generated as follows:

[0165]

[0166] Through the existing user feature data and the fake user feature data, a binary classification training task of the multi-service recommendation model is constructed. The set of existing user feature data is denoted as D r , each of its data labels is recorded as 1, and the set of forged user feature data is recorded as D f , the label of each data is recorded as 0. Then the objective function of model training is as follows:

[0167] The objective function trains all learnable parameters in CDEAttNN end-to-end.

[0168] The embodiment of the present invention adopts a data forgery identification learning strategy to train a multi-service recommendation model. Two types of methods related to the training method proposed in the embodiment of the present invention are data representation learning methods and adversarial training methods. The method proposed in this patent is different from the above two methods:

[0169] Compared with the existing data representation learning method, the existing data representation learning method mainly constructs the data similarity measurement criterion, directly trains the parameters of some modules of the multi-service recommendation model, and finally uses the trained part of the model parameters to directly bring it into the multi-service recommendation model. The subsequent multi-service recommendation model classification training process will keep these parameters unchanged, so it is relatively independent of the user data representation learning process. The embodiment of the present invention does not construct a similarity measurement criterion, but directly designs a new data forging method, which effectively identifies by directly training the multi-service recommendation model with forged data.

[0170] Compared with the adversarial training method, the data forgery in the embodiment of the present invention is based on static criteria, rather than adversarial samples generated by the adversarial training method that are unrecognizable to the multi-service recommendation model. The advantage of this is that user feature data belongs to tabular data, and general adversarial perturbations cannot maintain the structural characteristics of tabular data, thereby affecting the training effect. For example, discrete variable values ​​cannot form an effective embedding representation after a small perturbation.

[0171] This method can further optimize the training parameters of the multi-service recommendation model through feedback from the multi-service recommendation model, so that the multi-service recommendation model can identify the boundaries of the original user feature data while completing the target task, thereby better learning the user feature distribution and improving the performance of the model in the target task.

[0172] In summary, the CDEAtt based on the sparse attention mechanism proposed in the embodiment of the present invention can effectively solve the technical bottleneck of efficient extraction of potential associations caused by the lack of spatial and contextual associations in user feature data; and the embodiment of the present invention adopts a data forgery identification learning strategy to further improve the model performance of multi-service recommendation, thereby improving the accuracy of the model in predicting user preferences for multiple services.

[0173] like Figure 5 As shown, the embodiment of the present invention further provides a multi-service recommendation device, including:

[0174] An acquisition module 501 is used to input user feature data into a multi-service recommendation model to obtain the user's purchase confidence for each service or product output by the multi-service recommendation model;

[0175] The multi-service recommendation model optimizes the user's feature matrix using a sparse attention mechanism.

[0176] As an optional embodiment, the multi-service recommendation model includes:

[0177] An input processor, used to take user features as input and output a feature matrix for each user;

[0178] The neural network module based on the attention mechanism is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the vector of the corresponding row of the optimized feature representation matrix;

[0179] And, a classifier module is used to take the optimized feature representation matrix as input and output the purchase probability of the user for each business or product.

[0180] As an optional embodiment, the input processor is further configured to:

[0181] Construct an overall feature matrix based on the user features in the data set;

[0182] A feature matrix for each user feature is determined according to the type of the user feature and the overall feature matrix.

[0183] As an optional embodiment, the neural network module based on the attention mechanism includes: a multi-head attention mechanism module and a feed-forward neural network FFN block module;

[0184] The multi-head attention mechanism module is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the optimized feature representation matrix;

[0185] The FFN block module is used to transform and optimize the row vectors of the feature representation matrix output by the multi-head attention mechanism module through a fully connected network to obtain the vectors of the corresponding rows of the optimized feature representation matrix.

[0186] As an optional embodiment, the device further includes:

[0187] Configuration unit, used to configure the first trainable parameter matrix W Q , the second trainable parameter matrix W K , the third trainable parameter matrix W V and a sparse matrix M;

[0188] Construction unit for constructing the vector CDEAtt(PX i ,W Q ,W K ,W V ,M); where X i represents the user's feature matrix; W Q represents the first trainable parameter matrix; W K represents the second trainable parameter matrix; W V represents the training parameter matrix of the third lesson; M represents a sparse matrix; P represents X i as input and output R n×nThe output of P satisfies the following conditions:

[0189] P kj ≥0,k,j=1,…,n; and

[0190] As an optional embodiment, the sparse matrix M has the following form:

[0191]

[0192] Among them, M1, M2 and M3 are pre-defined block diagonal sparse matrices respectively; M1 corresponds to the first type of variables, and the size of M1 is the same as the number of the first type of variables; M2 corresponds to the second type of continuous variables, and the size of M2 is the same as the number of the second type of variables; M3 corresponds to the third type of variables, and the size of M3 is the same as the number of the third type of variables; -inf represents an infinitesimal quantity.

[0193] As an optional embodiment, the neural network P has the following form:

[0194]

[0195] Among them, P1, P2 and P3 are matrices respectively; P1 corresponds to the first type of variables, and the size of P1 is the same as the number of the first type of variables; P2 corresponds to the second type of variables, and the size of P2 is the same as the number of the second type of variables; P3 corresponds to the third type of variables, and the size of P3 is the same as the number of the third type of variables.

[0196] As an optional embodiment, the multi-head attention mechanism module is further used to:

[0197] Based on the vector of the sparse attention mechanism, the calculation formula for determining the multi-head attention mechanism module is:

[0198] MultiHeadCDEAtt=Concat(CDEAtt 1 ,…,CDEAtt C )W o ;

[0199] Among them, Concat means that the vectors are concatenated according to the last dimension; CDEAtt 1 ,…,CDEAtt C Represents the vector of each sparse attention mechanism; W o Represents the preset matrix corresponding to the multi-head attention mechanism;

[0200]

[0201] As an optional embodiment, the classifier module is further used for:

[0202] The concatenated vector of the corresponding row of the optimized feature representation matrix output by the multi-head attention mechanism module is used as input, and the output vector O with a length of C is i ;

[0203] Among them, O i The kth component of indicates whether the user purchases the kth service or product.

[0204] As an optional embodiment, the device further includes:

[0205] A training unit, configured to perform adversarial training on the multi-service recommendation model by adopting an adversarial contrast learning strategy to obtain the multi-service recommendation model; the adversarial contrast learning strategy includes: a target task for multi-task recommendation and a data forgery identification learning task;

[0206] The target task of the multi-task recommendation is used to predict the purchase confidence of each business or product based on real user characteristics;

[0207] The data forgery identification learning task is used to train a multi-service recommendation model to identify real user features by forging user features.

[0208] As an optional embodiment, the device further includes:

[0209] A construction unit, used to construct a forged user feature based on random column concatenation of different user features;

[0210] A training construction unit, used to construct a binary classification training task of a multi-service recommendation model by performing model training on real user features and the forged user features; the binary classification training task includes: a target task of multi-task recommendation and a data forgery identification learning task;

[0211] Among them, the objective function of model training is:

[0212]

[0213] Among them, CDEAttNN represents the multi-service recommendation model; D L Represents a set of real user features; D f represents a set of forged user features; the objective function performs end-to-end training on all learnable parameters in CDEAttNN.

[0214] The embodiment of the present invention optimizes the standard attention mechanism in the multi-service recommendation model, and uses sparse attention mechanism learning to avoid the impact of feature redundancy and invalid information caused by associating unrelated features on the multi-service recommendation model; and through the convex combination of the feature representation matrix in the multi-service recommendation model of the present application, the structural characteristics of the sparse matrix can be fully utilized to explore the potential associations between different columns, while avoiding information redundancy and interference caused by unnecessary associations, thereby improving model performance.

[0215] It should be noted that the multi-service recommendation device provided in the embodiment of the present invention is a device capable of executing the above-mentioned multi-service recommendation method. All embodiments of the above-mentioned multi-service recommendation method are applicable to the device and can achieve the same or similar beneficial effects, which will not be repeated here.

[0216] like Figure 6 As shown, an embodiment of the present invention further provides a multi-service recommendation device, including a processor 600 and a transceiver 610, wherein the transceiver 610 receives and sends data under the control of the processor 600, and the processor 600 is used to perform the following operations:

[0217] Inputting the user feature data into the multi-service recommendation model, and obtaining the user's purchase confidence for each service or product output by the multi-service recommendation model;

[0218] The multi-service recommendation model optimizes the user's feature matrix using a sparse attention mechanism.

[0219] As an optional embodiment, the multi-service recommendation model includes:

[0220] An input processor, used to take user features as input and output a feature matrix for each user;

[0221] The neural network module based on the attention mechanism is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the vector of the corresponding row of the optimized feature representation matrix;

[0222] And, a classifier module is used to take the optimized feature representation matrix as input and output the purchase probability of the user for each business or product.

[0223] As an optional embodiment, the input processor is further configured to:

[0224] Construct an overall feature matrix based on the user features in the data set;

[0225] A feature matrix for each user feature is determined according to the type of the user feature and the overall feature matrix.

[0226] As an optional embodiment, the neural network module based on the attention mechanism includes: a multi-head attention mechanism module and a feed-forward neural network FFN block module;

[0227] The multi-head attention mechanism module is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the optimized feature representation matrix;

[0228] The FFN block module is used to transform and optimize the row vectors of the feature representation matrix output by the multi-head attention mechanism module through a fully connected network to obtain the vectors of the corresponding rows of the optimized feature representation matrix.

[0229] As an optional embodiment, the processor is further configured to perform the following operations:

[0230] Configure the first trainable parameter matrix W Q , the second trainable parameter matrix W K , the third trainable parameter matrix W V and a sparse matrix M;

[0231] Construct the vector CDEAtt(PX i ,W Q ,W K ,W V ,M); where X i represents the user's feature matrix; W Q represents the first trainable parameter matrix; W K represents the second trainable parameter matrix; W V represents the training parameter matrix of the third lesson; M represents a sparse matrix; P represents X i as input and output R n×n The output of P satisfies the following conditions:

[0232] P kj ≥0,k,j=1,…,n; and

[0233] As an optional embodiment, the sparse matrix M has the following form:

[0234]

[0235] Among them, M1, M2 and M3 are pre-defined block diagonal sparse matrices respectively; M1 corresponds to the first type of variables, and the size of M1 is the same as the number of the first type of variables; M2 corresponds to the second type of continuous variables, and the size of M2 is the same as the number of the second type of variables; M3 corresponds to the third type of variables, and the size of M3 is the same as the number of the third type of variables; -inf represents an infinitesimal quantity.

[0236] As an optional embodiment, the neural network P has the following form:

[0237]

[0238] Among them, P1, P2 and P3 are matrices respectively; P1 corresponds to the first type of variables, and the size of P1 is the same as the number of the first type of variables; P2 corresponds to the second type of variables, and the size of P2 is the same as the number of the second type of variables; P3 corresponds to the third type of variables, and the size of P3 is the same as the number of the third type of variables.

[0239] As an optional embodiment, the multi-head attention mechanism module is further used to:

[0240] Based on the vector of the sparse attention mechanism, the calculation formula for determining the multi-head attention mechanism module is:

[0241] MultiHeadCDEAtt=Concat(CDEAtt 1 ,…,CDEAtt C )W o ;

[0242] Among them, Concat means that the vectors are concatenated according to the last dimension; CDEAtt 1 ,…,CDEAtt C Represents the vector of each sparse attention mechanism; W o Represents the preset matrix corresponding to the multi-head attention mechanism;

[0243]

[0244] As an optional embodiment, the classifier module is further used for:

[0245] The concatenated vector of the corresponding row of the optimized feature representation matrix output by the multi-head attention mechanism module is used as input, and the output vector O with a length of C is i ;

[0246] Among them, O i The kth component of indicates whether the user purchases the kth service or product.

[0247] As an optional embodiment, the processor is further configured to perform the following operations:

[0248] Adopting an adversarial contrast learning strategy to conduct adversarial training on the multi-service recommendation model to obtain the multi-service recommendation model; the adversarial contrast learning strategy includes: a target task for multi-task recommendation and a data forgery identification learning task;

[0249] The target task of the multi-task recommendation is used to predict the purchase confidence of each business or product based on real user characteristics;

[0250] The data forgery identification learning task is used to train a multi-service recommendation model to identify real user features by forging user features.

[0251] As an optional embodiment, the processor is further configured to perform the following operations:

[0252] Construct fake user features based on random column concatenation of different user features;

[0253] By training models based on real user features and the forged user features, a binary classification training task of a multi-service recommendation model is constructed; the binary classification training task includes: a target task for multi-task recommendation and a data forgery identification learning task;

[0254] Among them, the objective function of model training is:

[0255]

[0256] Among them, CDEAttNN represents the multi-service recommendation model; D L Represents a set of real user features; D f represents a set of forged user features; the objective function performs end-to-end training on all learnable parameters in CDEAttNN.

[0257] The embodiment of the present invention optimizes the standard attention mechanism in the multi-service recommendation model, and uses sparse attention mechanism learning to avoid the impact of feature redundancy and invalid information caused by associating unrelated features on the multi-service recommendation model; and through the convex combination of the feature representation matrix in the multi-service recommendation model of the present application, the structural characteristics of the sparse matrix can be fully utilized to explore the potential associations between different columns, while avoiding information redundancy and interference caused by unnecessary associations, thereby improving model performance.

[0258] It should be noted that the multi-service recommendation device provided in the embodiment of the present invention is a device capable of executing the above-mentioned multi-service recommendation method. All embodiments of the above-mentioned multi-service recommendation method are applicable to the device and can achieve the same or similar beneficial effects, which will not be repeated here.

[0259] An embodiment of the present invention further provides a multi-service recommendation device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the various processes in the multi-service recommendation embodiment described above are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0260] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, each process in the embodiment of the multi-service recommendation method described above is implemented, and the same technical effect can be achieved. To avoid repetition, it is not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0261] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0262] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A device that specifies functions in one or more processes and / or one or more blocks.

[0263] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable storage medium produce a paper product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0264] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that the computer or other programmable device executes a series of operating steps to produce a computer-implemented process, thereby providing instructions executed on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0265] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-service recommendation method, characterized in that: include: Inputting the user feature data into the multi-service recommendation model, and obtaining the user's purchase confidence for each service or product output by the multi-service recommendation model; The multi-service recommendation model optimizes the user's feature matrix using a sparse attention mechanism.

2. The method according to claim 1, characterized in that The multi-service recommendation model includes: An input processor, used to take user features as input and output a feature matrix for each user; The neural network module based on the attention mechanism is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the vector of the corresponding row of the optimized feature representation matrix; And, a classifier module is used to take the optimized feature representation matrix as input and output the user's purchase probability for each business or product.

3. The method according to claim 2, characterized in that The input processor is further configured to: Construct an overall feature matrix based on the user features in the data set; A feature matrix for each user feature is determined according to the type of user feature and the overall feature matrix.

4. The method according to claim 2, characterized in that: The neural network module based on the attention mechanism includes: a multi-head attention mechanism module and a feed-forward neural network FFN block module; The multi-head attention mechanism module is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the optimized feature representation matrix; The FFN block module is used to transform and optimize the row vectors of the feature representation matrix output by the multi-head attention mechanism module through a fully connected network to obtain the vectors of the corresponding rows of the optimized feature representation matrix.

5. The method according to claim 4, characterized in that The method further comprises: Configure the first trainable parameter matrix W Q , the second trainable parameter matrix W K , the third trainable parameter matrix W V and a sparse matrix M; Construct the vector CDEAtt(PX i ,W Q ,W K ,W V ,M); where X i represents the user's feature matrix; W Q represents the first trainable parameter matrix; W K represents the second trainable parameter matrix; W V represents the training parameter matrix of the third lesson; M represents a sparse matrix; P represents X i as input and output R n×n The output of P satisfies the following conditions: P kj ≥0,k,j=1,…,n; and 6. The method according to claim 5, characterized in that The sparse matrix M has the following form: Among them, M1, M2 and M3 are pre-defined block diagonal sparse matrices respectively; M1 corresponds to the first type of variables, and the size of M1 is the same as the number of the first type of variables; M2 corresponds to the second type of continuous variables, and the size of M2 is the same as the number of the second type of variables; M3 corresponds to the third type of variables, and the size of M3 is the same as the number of the third type of variables; -inf represents an infinitesimal quantity.

7. The method according to claim 5, characterized in that The neural network P has the following form: Among them, P1, P2 and P3 are matrices respectively; P1 corresponds to the first type of variables, and the size of P1 is the same as the number of the first type of variables; P2 corresponds to the second type of variables, and the size of P2 is the same as the number of the second type of variables; P3 corresponds to the third type of variables, and the size of P3 is the same as the number of the third type of variables.

8. The method according to claim 5, characterized in that The multi-head attention mechanism module is further used to: Based on the vector of the sparse attention mechanism, the calculation formula for determining the multi-head attention mechanism module is: MultiHeadCDEAtt=Concat(CDEAtt 1 ,…,CDEAtt C )W o ; Among them, Concat means that the vectors are concatenated according to the last dimension; CDEAtt 1 ,…,CDEAtt C Represents the vector of each sparse attention mechanism; W o Represents the preset matrix corresponding to the multi-head attention mechanism; 9. The method according to claim 8, characterized in that The classifier module is further used to: The concatenated vector of the corresponding row of the optimized feature representation matrix output by the multi-head attention mechanism module is used as input, and the output vector O with a length of C is i ; Among them, O i The kth component of indicates whether the user purchases the kth service or product.

10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: Adopting an adversarial contrast learning strategy to conduct adversarial training on the multi-service recommendation model to obtain the multi-service recommendation model; the adversarial contrast learning strategy includes: a target task for multi-task recommendation and a data forgery identification learning task; The target task of the multi-task recommendation is used to predict the purchase confidence of each business or product based on real user characteristics; The data forgery identification learning task is used to train a multi-service recommendation model to identify real user features by forging user features.

11. The method according to claim 10, characterized in that The method further comprises: Construct fake user features based on random column concatenation of different user features; By training models based on real user features and the forged user features, a binary classification training task of a multi-service recommendation model is constructed; the binary classification training task includes: a target task for multi-task recommendation and a data forgery identification learning task; Among them, the objective function of model training is: Among them, CDEAttNN represents the multi-service recommendation model; D L Represents a set of real user features; D f represents a set of forged user features; the objective function performs end-to-end training on all learnable parameters in CDEAttNN.

12. A multi-service recommendation device, characterized in that: include: An acquisition module, used to input user feature data into a multi-service recommendation model, and obtain the user's purchase confidence for each service or product output by the multi-service recommendation model; The multi-service recommendation model optimizes the user's feature matrix using a sparse attention mechanism.

13. A multi-service recommendation device, comprising a processor and a transceiver, wherein the transceiver receives and sends data under the control of the processor, characterized in that: The processor is configured to perform the following operations: Inputting the user feature data into the multi-service recommendation model, and obtaining the user's purchase confidence for each service or product output by the multi-service recommendation model; The multi-service recommendation model optimizes the user's feature matrix using a sparse attention mechanism.

14. The device according to claim 13, characterized in that The multi-service recommendation model includes: An input processor, used to take user features as input and output a feature matrix for each user; The neural network module based on the attention mechanism is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the vector of the corresponding row of the optimized feature representation matrix; And, a classifier module is used to take the optimized feature representation matrix as input and output the user's purchase probability for each business or product.

15. The device according to claim 14, characterized in that The input processor is further configured to: Construct an overall feature matrix based on the user features in the data set; A feature matrix for each user feature is determined according to the type of user feature and the overall feature matrix.

16. The device according to claim 14, characterized in that The neural network module based on the attention mechanism includes: a multi-head attention mechanism module and a feed-forward neural network FFN block module; The multi-head attention mechanism module is used to take the feature matrix of each user as input, optimize the feature matrix of each user using the sparse attention mechanism, and output the optimized feature representation matrix; The FFN block module is used to transform and optimize the row vectors of the feature representation matrix output by the multi-head attention mechanism module through a fully connected network to obtain the vectors of the corresponding rows of the optimized feature representation matrix.

17. The device according to claim 16, characterized in that The processor is further configured to perform the following operations: Configure the first trainable parameter matrix W Q , the second trainable parameter matrix W K , the third trainable parameter matrix W V and a sparse matrix M; Construct the vector CDEAtt(PX i ,W Q ,W K ,W V ,M); where X i represents the user's feature matrix; W Q represents the first trainable parameter matrix; W K represents the second trainable parameter matrix; W V represents the training parameter matrix of the third lesson; M represents a sparse matrix; P represents X i as input and output R n×n The output of P satisfies the following conditions: P kj ≥0,k,j=1,…,n; and 18. The device according to claim 17, characterized in that The sparse matrix M has the following form: Among them, M1, M2 and M3 are pre-defined block diagonal sparse matrices respectively; M1 corresponds to the first type of variables, and the size of M1 is the same as the number of the first type of variables; M2 corresponds to the second type of continuous variables, and the size of M2 is the same as the number of the second type of variables; M3 corresponds to the third type of variables, and the size of M3 is the same as the number of the third type of variables; -inf represents an infinitesimal quantity.

19. The device according to claim 17, characterized in that The neural network P has the following form: Among them, P1, P2 and P3 are matrices respectively; P1 corresponds to the first type of variables, and the size of P1 is the same as the number of the first type of variables; P2 corresponds to the second type of variables, and the size of P2 is the same as the number of the second type of variables; P3 corresponds to the third type of variables, and the size of P3 is the same as the number of the third type of variables.

20. The device according to claim 17, characterized in that The multi-head attention mechanism module is further used to: Based on the vector of the sparse attention mechanism, the calculation formula for determining the multi-head attention mechanism module is: MultiHeadCDEAtt=Concat(CDEAtt 1 ,…,CDEAtt C )W o ; Among them, Concat means that the vectors are concatenated according to the last dimension; CDEAtt 1 ,…,CDEAtt C Represents the vector of each sparse attention mechanism; W o Represents the preset matrix corresponding to the multi-head attention mechanism; 21. The device according to claim 20, characterized in that The classifier module is further used to: The concatenated vector of the corresponding row of the optimized feature representation matrix output by the multi-head attention mechanism module is used as input, and the output vector O with a length of C is i ; Among them, O i The kth component of indicates whether the user purchases the kth service or product.

22. The device according to any one of claims 13 to 21, characterized in that The processor is further configured to perform the following operations: Adopting an adversarial contrast learning strategy to conduct adversarial training on the multi-service recommendation model to obtain the multi-service recommendation model; the adversarial contrast learning strategy includes: a target task for multi-task recommendation and a data forgery identification learning task; The target task of the multi-task recommendation is used to predict the purchase confidence of each business or product based on real user characteristics; The data forgery identification learning task is used to train a multi-service recommendation model to identify real user features by forging user features.

23. The device according to claim 22, characterized in that The processor is further configured to perform the following operations: Construct fake user features based on random column concatenation of different user features; By training models based on real user features and the forged user features, a binary classification training task for a multi-service recommendation model is constructed; The binary classification training tasks include: a target task for multi-task recommendation and a data forgery identification learning task; Among them, the objective function of model training is: Among them, CDEAttNN represents the multi-service recommendation model; D L Represents a set of real user features; D f represents a set of forged user features; the objective function performs end-to-end training on all learnable parameters in CDEAttNN.

24. A multi-service recommendation device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the multi-service recommendation method according to any one of claims 1 to 11 is implemented.

25. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the multi-service recommendation method according to any one of claims 1 to 11 are implemented.

Images