Combined recommendation method and device for self-attention mechanism of low-purchase-degree user
Through the self-attention mechanism and user behavior preference classification model, combined with the multi-head self-attention layer and the full connection layer, the problem of traditional recommendation methods being poor for low-purchase users is solved, and better capture and improve user behavior preferences and Recommended improvements.
Patent Information
- Application Number
- CN202411421586.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional recommendation methods are not ideal for low-purchase users, and cannot effectively capture complex relationships and changes in user behavior preferences, resulting in poor recommendation results and failure to make full use of user characteristics and behavior data.
The self-attention mechanism is used to combine the user behavior preference classification model of the multi-head self-attention layer and the fully connected layer. It is weighted to represent the interactive behavior matrix of users and product characteristics, and a product order diagram and bundled combination are constructed, and a recommendation list is generated using the bundle search algorithm. .
It improves the recommendation effect for low-purchase users, dynamically learns user behavior preferences, generates more diverse and representative bundled combinations, and improves users' purchasing willingness and satisfaction.
Smart Images

Figure CN118967274A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing, and in particular to a combination recommendation method and device of a self-attention mechanism for low-purchasing users. Background Art
[0002] With the continuous expansion of the Internet market and increasingly fierce competition, the traditional recommendation of a single Internet product can no longer meet the increasingly diversified and combined needs of users. For this reason, the bundling combination of Internet products has emerged as a new sales model. Internet products are combined and recommended to users in the form of a bundled combination, providing users with a variety of product combinations.
[0003] However, after analyzing the user's characteristic information, it was found that the majority of the user group is low-purchasing Internet users. This type of user has significant characteristics: product purchase data is generally scarce or even almost missing, but click, favorite and add-to-cart behavior data is relatively rich. For this type of user, traditional recommendation methods are not very effective.
[0004] The low purchase rate problem is a classic problem in recommendation algorithms and has been plaguing the development of recommendation technology. Since traditional recommendations often rely on users' purchase data, the recommendation effect for users with low or even zero purchase rates is not ideal. However, these users account for a large proportion of the total number of users, and with the continuous advancement of technology, they have received more and more attention.
[0005] In some past studies, in order to get rid of the recommendation dilemma caused by the low purchase degree problem, explorations have been carried out from multiple angles. For example, Volkovs et al. used supplementary information efficiently, and regarded information other than ID information, such as user gender, age, characteristics and other attribute features as information sources, and made full use of these attribute features to adjust the model during model training so that the model no longer relies too much on purchase interaction information. When the low purchase degree problem is refined, it is further divided into two directions: low purchase degree users and low purchase degree products. In the direction of low purchase degree products, Pan et al. proposed to use other features such as items, and use a generator to simulate the generation of purchase interaction information and assign initialization values. This method trains the generator through existing items. For newly added products, all use this generator to predict the initialization value, and then conduct subsequent training and prediction based on the initialization value.
[0006] In subsequent studies, it was found that although there is only limited interaction data for low-purchase items, more relevant information can be mined through knowledge graphs. The knowledge graph method proposed by Ji et al. can also be applied to the low-purchase problem, using rich associated information to resolve the low-purchase problem. However, the angle of solving the problem is not always consistent. The cross-domain recommendation technology proposed by Zhu et al. has also become another key technology to solve the low-purchase problem, and the subsequent research on multi-behavior problems is also to effectively solve the low-purchase problem. With the continuous deepening of research, a detailed analysis of low-purchase users found that some low-purchase users just have no purchase data, but they have rich behavior data, so multi-behavior data has also become a key point. Through the analysis of multi-behavior data, the user's potential interest preferences can be obtained, thereby alleviating the recommendation dilemma caused by the low-purchase problem.
[0007] Based on the above analysis, the core idea to solve the problem of low purchase rate is to utilize other data information besides purchase data. User feature information, product feature information and user behavior information become the key elements to solve this problem.
[0008] However, the above methods do not really target low-purchase users in the Internet field. At the same time, they are not combined with the self-attention mechanism, which makes it impossible to better capture the complex relationships and changes of user behavior preferences.
[0009] For Internet companies, the low-purchase user group represents a potential market that has not yet been fully developed. Therefore, providing suitable product bundles for such low-purchase users with rich behavioral data has become an urgent problem to be solved. By analyzing the behavioral characteristics and preferences of these users, their willingness to buy and satisfaction can be increased, thereby improving the retention rate and conversion rate of such Internet users. Through in-depth analysis of a large number of low-purchase users, Internet companies can discover market gaps and shortcomings of competitors, provide companies with innovative and differentiated competitive strategies, and enhance market competitiveness. Summary of the invention
[0010] The purpose of this application is to propose a combined recommendation method and device of the self-attention mechanism for low-purchasing users in response to the above-mentioned technical problems.
[0011] In a first aspect, the present invention provides a combined recommendation method of a self-attention mechanism for low-purchase users, comprising the following steps:
[0012] Obtain the behavior interaction information of each user in the user group and obtain the behavior interaction order set according to the time when the interaction behavior occurs. Classify and summarize the behavior interaction order set according to different interaction behaviors to obtain the product order subsets of each user under different interaction behaviors. The interaction behaviors include click, favorite and add to cart. Superimpose the product order subsets of all users under different interaction behaviors to obtain the product order graph generated by the user group under each interaction behavior. Merge the product order graphs generated by the user group under all interaction behaviors to obtain the overall product order graph.
[0013] A user behavior preference classification model is constructed and trained to obtain a trained user behavior preference classification model, wherein the user behavior preference classification model includes a concatenation layer, a multi-head self-attention layer, and a fully connected layer connected in sequence;
[0014] Obtain a user feature vector corresponding to one of the users in the user group and a product feature vector corresponding to each Internet product and input them into a trained user behavior preference classification model. Input the user feature vector and the product feature vector into the concatenation layer to obtain a behavior interaction matrix of one of the users for each Internet product. Input the behavior interaction matrix of one of the users for each Internet product into the multi-head self-attention layer to obtain a weighted behavior feature vector. Input the weighted behavior feature vector into the fully connected layer to obtain a behavior preference classification result of one of the users for each Internet product.
[0015] The starting point for generating a bundle is determined based on the behavioral interaction order set of one of the users in the user group and the behavioral preference classification results of all Internet products. A beam search algorithm is used to search for extended nodes from the starting point to form a bundle based on the product order graph and the user's behavioral preference classification results. A bundle list is obtained. The bundles in the bundle list are screened to obtain a recommendation list.
[0016] As a preferred embodiment, the behavior interaction sequence set of one of the users in the user group is: , , Represents a subset of product order generated by the click behavior of one of the users in the user group; Represents a subset of product order generated by the collection behavior of one user in the user group; It represents the product order subset generated by the purchase behavior of one of the users in the user group, where the order is determined by the time when the behavior occurs; the product order diagram of the user group under different interactive behaviors is expressed as , , =1,2,3, =1 indicates click behavior, =2 indicates collection behavior, =3 indicates additional purchase behavior, where: , Represents the node set in the product sequence graph under different interaction behaviors. The i-th Internet product Represents the i-th node , i=1,2,…,n, represents the edge set in the product order graph under different interaction behaviors, , , g = 1, 2, …, m, Represented as slave nodes under different interaction behaviors Departure to the node And the weight is The directed edges of The calculation formula is as follows:
[0017] ;
[0018] ;
[0019] in, represents the kth user in the user group, c represents the total number of users in the user group, Indicates the first Internet product and the jth Internet product The weight corresponding to performing one of the interactive behaviors, for The indicator function of Click on the Internet product ,but ,otherwise , for The indicator function of Click on the Internet product ,but ,otherwise .
[0020] Preferably, the product sequence graphs generated by the user group under all interactive behaviors are merged to obtain a product sequence master graph, which specifically includes:
[0021] The node set of the product order graph is the union of the node sets of the product order graphs generated under all interaction behaviors, the edge set of the product order graph is the union of the edge sets of the product order graphs generated under all interaction behaviors, and the weight of the directed edges in the edge set of the product order graph is the sum of the weights of the directed edges in the edge set of the product order graph generated under each interaction behavior.
[0022] Preferably, a user feature vector corresponding to one of the users in the user group and a product feature vector corresponding to each Internet product are obtained and input into a trained user behavior preference classification model, the user feature vector and the product feature vector are input into a concatenation layer, and a behavior interaction matrix of one of the users for each Internet product is obtained, and the behavior interaction matrix of one of the users for each Internet product is input into a multi-head self-attention layer to obtain a weighted behavior feature vector, and the weighted behavior feature vector is input into a fully connected layer to obtain a behavior preference classification result of one of the users for each Internet product, specifically including:
[0023] The product feature vector and user feature vector are input into the concatenation layer, and are first concatenated according to different interaction behaviors to obtain the interaction behavior vector corresponding to the different interaction behaviors of the k-th user on the i-th Internet product. The calculation formula is as follows:
[0024] ;
[0025] in, =1,2,3, =1 indicates click behavior, =2 indicates collection behavior, =3 indicates additional purchase behavior, represents the product feature vector of the ith Internet product. The product feature vector is a vector composed of the quantified values of product features. Product features include product type, price, warranty period, product status, and sales status. represents the user feature vector of the kth user. The user feature vector is a vector composed of the quantized values of user features. User features include age, gender, occupation, marital status, and number of children. Indicates that and Stitch together according to the same interactive behavior;
[0026] Combine the interaction behavior vectors of each interaction behavior generated by the k-th user on the i-th Internet product to obtain the behavior interaction matrix of the k-th user on the i-th Internet product , as shown below:
[0027] ;
[0028] The behavior interaction matrix of the kth user for the i-th Internet product is input into the multi-head self-attention layer, and calculations are performed on each self-attention head to obtain the output of each self-attention head. The calculation formula is as follows:
[0029] ;
[0030] ;
[0031] ;
[0032] ;
[0033] in, represents the output of the ηth self-attention head in the multi-head self-attention layer, η=1,2,…,o, Represents the function corresponding to the self-attention mechanism, represents the behavioral interaction matrix of the kth user on the i-th Internet product, denote the learnable matrices corresponding to the nth query vector, the nth key vector, and the nth value vector, respectively, , and represents the nth query vector, the nth key vector and the nth value vector, and D represents The dimension of , T represents transpose;
[0034] The outputs of each self-attention head are concatenated to obtain the weighted behavioral feature vector of the kth user for the i-th Internet product , the formula is as follows:
[0035] ;
[0036] The weighted behavioral feature vector of the k-th user for the i-th Internet product is input into the fully connected layer to obtain the category probability matrix. The calculation formula is as follows:
[0037] ;
[0038] in, represents the class probability vector, represents the weight matrix, express The activation function, b represents the bias, each element in the category probability vector represents the probability of the k-th user generating each interactive behavior on the ith Internet product, and the interactive behavior corresponding to the maximum value of all elements in the category probability vector is taken as the behavioral preference classification result of the k-th user on the ith Internet product.
[0039] Preferably, determining the starting point for bundling combination generation based on the behavior interaction order set of one of the users in the user group and the behavior preference classification results of all Internet products specifically includes:
[0040] The behavior preference classification results of the kth user for all Internet products are classified according to different interactive behaviors, and under each interactive behavior, the probability of the kth user generating this interactive behavior for each Internet product is sorted from large to small, and the top h Internet products are selected to form a product set corresponding to each interactive behavior. , =1,2,3, =1 indicates click behavior, =2 indicates collection behavior, =3 indicates additional purchase behavior;
[0041] From the product collection corresponding to the add-to-cart behavior Determine the starting point of bundle combination generation;
[0042] In response to determining that there is an Internet product that generates an add-to-purchase behavior in the behavior interaction sequence set of the kth user, determine the Internet product that most recently generated the add-to-purchase behavior from the behavior interaction sequence set of the kth user according to the most recent add-to-purchase time of the kth user in the time dimension, and calculate the Internet product that most recently generated the add-to-purchase behavior and the product set corresponding to the add-to-purchase behavior The similarity of Internet products in the product set corresponding to the add-to-cart behavior The Internet product with the highest similarity to the Internet product that generated the most recent purchase behavior is selected as the starting point for generating the bundle combination for the kth user;
[0043] In response to determining that there is no Internet product that generates the add-to-cart behavior in the behavior interaction sequence set of the kth user, the Internet product that most recently generated the collection behavior is determined from the behavior interaction sequence set of the kth user in the time dimension according to the most recent collection time of the kth user, and the Internet product that most recently generated the collection behavior and the product set corresponding to the collection behavior are calculated. The similarity of Internet products in the product set corresponding to the collection behavior The Internet product with the highest similarity to the Internet product that generated the most recent collection behavior is selected as the starting point for generating the bundle combination of the kth user;
[0044] In response to determining that there is no Internet product that generates the add-to-cart and favorite behaviors in the behavior interaction sequence set of the kth user, the Internet product that most recently generated the click behavior is determined from the behavior interaction sequence set of the kth user in the time dimension according to the most recent click time of the kth user, and the Internet product that most recently generated the click behavior and the product set corresponding to the click behavior are calculated. The similarity of Internet products in the set of products corresponding to the click behavior The Internet product with the highest similarity to the Internet product that generated the most recent click behavior is selected as the starting point for generating the bundle combination of the kth user.
[0045] As a preferred method, based on the product sequence diagram and the user's behavior preference classification results, a beam search algorithm is used to search for expansion nodes from the starting point to form a bundle combination, obtain a bundle combination list, and screen the bundle combinations in the bundle combination list to obtain a recommendation list, which specifically includes:
[0046] S41, initialization, starting from the starting point generated by the bundle combination of the kth user, and taking the starting point as the current candidate sequence;
[0047] S42, expanding the current candidate sequence, selecting each adjacent node that can be reached from the last node of the current candidate sequence from the product sequence general graph as an expansion node;
[0048] S43, calculating the jump probability between the last node of the current candidate sequence and each extended node, as shown below:
[0049] ;
[0050] ;
[0051] in, Indicates the last node of the current candidate sequence With extension nodes The jump probability between represents the path from one node to another node. Indicates the last node in the product sequence graph from the current candidate sequence Departure to the expansion node The weight of the directed edge of Indicates the last node of the current candidate sequence in the product sequence graph under different interaction behaviors Departure to the expansion node The weight of the directed edge of Represents a collection of expansion nodes;
[0052] S44, selecting beam width parameter nodes with the highest jump probability and not existing in the existing candidate sequence from the extended nodes, and forming the next candidate sequence with the current candidate sequence respectively;
[0053] S45, taking the next candidate sequence as the current candidate sequence and repeating steps S42-S44 until the next candidate sequence reaches the end condition, taking each candidate sequence as a bundle combination to form a bundle combination list for the kth user ;
[0054] Measure the importance of each Internet product in the bundled combination list of the kth user and obtain the frequency of each Internet product in the bundled combination list , as shown below:
[0055] ;
[0056] in, Indicates the eth Internet product List of bundle combinations for the kth user The frequency of occurrence in Indicates the eth Internet product List of bundle combinations for the kth user The number of occurrences in Represents the bundle combination list of the kth user The number of all Internet products in
[0057] According to frequency Sort in descending order, select the top U% of Internet product collection products as important products, and put them in the bundle combination list of the kth user Select the bundles that contain at least one important product to form the first list ;
[0058] The first list is calculated using the following formula The similarity of two different bundle combinations in:
[0059] ;
[0060] ;
[0061] ;
[0062] in, and Represents two different bundle combinations of the same user. and Represents two bundle combinations and The product feature vector corresponding to the Internet products in Represents two bundle combinations and The similarities between Internet products in Represents the similarity of two different bundles for the same user;
[0063] In the first list The bundle combinations whose similarity between two different bundle combinations is lower than the similarity threshold are selected to form the second list ;
[0064] Calculate the second list The Bundles The accumulated points , as shown below:
[0065] ;
[0066] ;
[0067] in, represents the kth user, Denotes the δth bundle combination Any two Internet products and The weight between represents the δth bundle combination under different interaction behaviors Middle slave node Departure to the node The weights of the directed edges of ;
[0068] For the second list All bundles in The accumulated points are sorted from large to small, and the top T bundle combinations are selected to form a recommendation list .
[0069] In a second aspect, the present invention provides a combined recommendation device of a self-attention mechanism for low-purchase users, comprising:
[0070] A product order graph generation module is configured to obtain the behavior interaction information of each user in the user group and obtain a behavior interaction order set according to the time when the interaction behavior occurs, classify and summarize the behavior interaction order set according to different interaction behaviors, obtain a product order subset of each user under different interaction behaviors, and the interaction behaviors include click, favorite and add to cart, respectively superimpose the product order subsets of all users under different interaction behaviors to obtain a product order graph generated by the user group under each interaction behavior, and merge the product order graphs generated by the user group under all interaction behaviors to obtain a product order graph;
[0071] A model building module is configured to build and train a user behavior preference classification model to obtain a trained user behavior preference classification model, wherein the user behavior preference classification model includes a concatenation layer, a multi-head self-attention layer, and a fully connected layer connected in sequence;
[0072] The behavior preference prediction module is configured to obtain a user feature vector corresponding to one of the users in the user group and a product feature vector corresponding to each Internet product and input them into a trained user behavior preference classification model, input the user feature vector and the product feature vector into a concatenation layer, obtain a behavior interaction matrix of one of the users for each Internet product, input the behavior interaction matrix of one of the users for each Internet product into a multi-head self-attention layer, obtain a weighted behavior feature vector, input the weighted behavior feature vector into a fully connected layer, and obtain a behavior preference classification result of one of the users for each Internet product;
[0073] The screening module is configured to determine the starting point for generating a bundle combination based on the behavioral interaction order set of one of the users in the user group and the behavioral preference classification results of all Internet products, and to use a beam search algorithm to search for extended nodes from the starting point to form a bundle combination based on the product order overall graph and the user's behavioral preference classification results, to obtain a bundle combination list, and to screen the bundle combinations in the bundle combination list to obtain a recommendation list.
[0074] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.
[0075] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.
[0076] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.
[0077] Compared with the prior art, the present invention has the following beneficial effects:
[0078] (1) The combined recommendation method of the self-attention mechanism for low-purchase users proposed in the present invention is aimed at the low-purchase user group. For each user, the product order generated by clicks, favorites and add-to-cart is constructed, and the three product orders of each user are superimposed to obtain the product order graphs of the three interactive behaviors of the user group. Finally, the product order graphs of the three interactive behaviors are superimposed to obtain the overall product order graph generated by the interactive behaviors. For low-purchase users, the purchase information is often too sparse to be applied to traditional recommendation algorithms. Therefore, the present invention constructs three product order graphs based on the user's interactive behavior information and superimposes them to analyze such users.
[0079] (2) The combined recommendation method of the self-attention mechanism for low-purchase users proposed in the present invention concatenates the user feature vector and the product feature vector to obtain the user-product behavior interaction matrix, which is input into the multi-head self-attention layer, and the self-attention weight is used to weight different behaviors, thereby learning the user's behavior towards different products. The self-attention mechanism can dynamically learn to assign different attention weights to different behavior interaction matrices to capture the complex relationships and changes of user behavior preferences, and better highlight the core behaviors that truly reflect user preferences.
[0080] (3) The self-attention mechanism combination recommendation method for low-purchase users proposed in this invention evaluates the similarity between bundle combinations based on the similarity metric of bulldozer distance, and selects bundle combinations with lower similarity and higher cumulative scores. This ensures that the recommendation list containing bundle combinations generated in the end is more diverse and representative, and avoids ignoring the overall similarity due to the differences in individual products. BRIEF DESCRIPTION OF THE DRAWINGS
[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0082] Figure 1 A flowchart of a combined recommendation method of a self-attention mechanism for low-purchase users according to an embodiment of the present application;
[0083] Figure 2 A schematic diagram of the construction and superposition of product sequence diagrams of three interactive behaviors of a combined recommendation method of a self-attention mechanism for low-purchasing users according to an embodiment of the present application;
[0084] Figure 3 A flowchart of a combined recommendation method of a self-attention mechanism for low-purchase users according to an embodiment of the present application;
[0085] Figure 4 A schematic diagram of a combined recommendation device of a self-attention mechanism for low-purchase users according to an embodiment of the present application;
[0086] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0087] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0088] Figure 1 The embodiment of the present application provides a combined recommendation method of the self-attention mechanism for low-purchase users, including the following steps:
[0089] S1, obtain the behavioral interaction information of each user in the user group and obtain the behavioral interaction order set according to the time when the interactive behavior occurs, classify and summarize the behavioral interaction order set according to different interactive behaviors, and obtain the product order subsets of each user under different interactive behaviors. The interactive behaviors include clicks, favorites, and add to cart. The product order subsets of all users under different interactive behaviors are superimposed separately to obtain the product order graph generated by the user group under each interactive behavior, and the product order graphs generated by the user group under all interactive behaviors are merged to obtain the overall product order graph.
[0090] In a specific embodiment, the behavior interaction sequence set of one user in the user group is: , , Represents a subset of product order generated by the click behavior of one of the users in the user group; Represents a subset of product order generated by the collection behavior of one user in the user group; It represents the product order subset generated by the purchase behavior of one of the users in the user group, where the order is determined by the time when the behavior occurs; the product order diagram of the user group under different interactive behaviors is expressed as , , =1,2,3, =1 indicates click behavior, =2 indicates collection behavior, =3 indicates additional purchase behavior, where: , Represents the node set in the product sequence graph under different interaction behaviors. The i-th Internet product Represents the i-th node , i=1,2,…,n, represents the edge set in the product order graph under different interaction behaviors, , , g = 1, 2, …, m, Represented as slave nodes under different interaction behaviors Departure to the node And the weight is The directed edges of The calculation formula is as follows:
[0091] ;
[0092] ;
[0093] in, represents the kth user in the user group, c represents the total number of users in the user group, Indicates the first Internet product and the jth Internet product The weight corresponding to performing one of the interactive behaviors, for The indicator function of Click on the Internet product ,but ,otherwise , for The indicator function of Click on the Internet product ,but ,otherwise .
[0094] In a specific embodiment, the product sequence graphs generated by the user group under all interactive behaviors are merged to obtain a product sequence master graph, which specifically includes:
[0095] The node set of the product order graph is the union of the node sets of the product order graphs generated under all interaction behaviors, the edge set of the product order graph is the union of the edge sets of the product order graphs generated under all interaction behaviors, and the weight of the directed edges in the edge set of the product order graph is the sum of the weights of the directed edges in the edge set of the product order graph generated under each interaction behavior.
[0096] Specifically, for users with low purchasing power, their purchasing information is often too sparse, and the purchasing information of this type of user group cannot be directly applied to the traditional recommendation model. Therefore, behavioral interaction information becomes the key to analyzing this type of users. Behavioral interaction information includes click behavior data, collection behavior data, and add-to-cart behavior data. Add-to-cart means adding to the shopping cart. In the process of analyzing behavioral interaction information, it is necessary to identify the user's behavior patterns and trends in order to deeply understand the intentions and motivations behind the user's behavior. The products of the embodiments of the present application take Internet insurance products as examples, and their corresponding interactive behaviors include clicks, collections, and add-to-cart, etc. Other Internet products can also be selected in other optional embodiments, which are not limited here. The product sequence diagram generated by the interactive behavior includes a variety of behavioral interaction information, which represents the user's potential preferences and behavioral habits. The product sequence diagram generated by the interactive behavior is an effective integration of user behavior data, reflecting the behavioral characteristics of Internet insurance users with low purchasing power.
[0097] When analyzing the user's behavioral interaction information, the embodiments of the present application need to consider different types of behavioral interaction information and perform separate processing and analysis for each type of behavioral interaction information. Therefore, in order to more comprehensively reflect the interests and needs of users and better generate a suitable combination of Internet insurance products for low-purchase users, it is considered to use the order of insurance products generated by interactive behaviors to construct a product sequence graph (the point set of the product sequence graph is the insurance products, and the edge set is the association weight between insurance products) to infer the user's preferences, predict their possible behaviors, and provide personalized recommendations and services to users based on this information.
[0098] First, the user's behavior interaction information is summarized and sorted according to the time when the interaction behavior occurs, and the user's behavior interaction order set is obtained. To classify and summarize, .in, Indicates the order of products generated by the user's click behavior; Indicates the order of products generated by the user's collection behavior; Indicates the order of products generated by the user's purchase behavior. For each user, there are three types of product orders generated by click, favorite, and purchase. For example, in chronological order, user A clicked on products , and , here, in order to simplify the processing, we save time and get the product order generated by the user's click behavior. For example, the product order of user A's click behavior =< , , >. The same processing can be applied to the collection and add-to-cart behaviors to obtain the product order of the user's collection behavior. And the order of products in the add-to-cart behavior The order of the three categories of products is different for each user.
[0099] After the above processing, the product order generated by the three interactive behaviors of each user is obtained For user groups, to analyze the associations between all insurance products generated by interactive behaviors, it is necessary to superimpose the product order generated by each interactive behavior. For example, for the order of clicked products, user A clicked on product After clicking on the product , user B also clicked on the product After clicking on the product , then construct point to The edge weight is 2. According to this method, the three different product orders of user clicks, favorites and purchases are superimposed respectively, and the product order graph generated by the user group click behavior can be obtained. , Product sequence diagram generated by collection behavior Product sequence diagram generated by the add-to-cart behavior However, considering the different importance of different interactive behaviors, different basic weights are assigned to two adjacent Internet products in the product sequence generated by different interactive behaviors of user groups. Based on this, different basic weights can be selected according to different interactive behaviors, so the differentiation of different interactive behaviors can be effectively constructed. When constructing the product sequence diagram of the three interactive behaviors of the user group, the basic weights After calculation, we get the product sequence diagram of the three interactive behaviors of the user group. By merging the three Internet product sequence diagrams, we can get the product sequence diagram of the interactive behavior of the user group. The merging method is: the point set of the total graph is the union of the point sets of each behavior product order graph; the edge set is the union of the edge sets of each behavior product order graph; if an edge appears in multiple Internet product order graphs, then the weight of this edge is the sum of the weights in each product order graph. According to the above method, the product order graph generated by the behavioral interaction of the entire low-purchase user group is formed. The product sequence diagram can reflect the relationship between insurance products, such as Figure 2 The nodes in the product sequence graph represent insurance products, and the edges and edge weights can represent the degree of association between insurance products.
[0100] S2, constructing and training a user behavior preference classification model to obtain a trained user behavior preference classification model, the user behavior preference classification model includes a concatenation layer, a multi-head self-attention layer and a fully connected layer connected in sequence.
[0101] Specifically, the generation of insurance product portfolios needs to consider two key factors at the same time: one is the correlation between insurance products; the other is the user's behavioral category preference for insurance products. Limited by the sparsity of purchase data, Internet insurance companies cannot effectively analyze user behavior preferences through the purchase data of low-purchase users. The interactive behavior information of low-purchase user groups has become an effective way to alleviate this problem. Figure 3 , establish a user behavior preference classification model, the input of which is the user feature vector of one of the users in the user group and the product feature vector corresponding to each Internet product, and the output is a category probability matrix, which contains the probability prediction of three interactive behaviors of one of the users on one of the Internet products, so the behavior preference classification result of one of the users in the user group on each Internet product can be determined, and the behavior preference classification result is one of the click behavior, collection behavior and purchase behavior. The loss function used in the training process of the user behavior preference classification model is the cross entropy loss function (Cross Entropy Loss), which is an existing loss function, and its mathematical formula is not repeated here.
[0102] S3, obtain the user feature vector corresponding to one of the users in the user group and the product feature vector corresponding to each Internet product and input them into the trained user behavior preference classification model, input the user feature vector and the product feature vector into the concatenation layer, obtain the behavior interaction matrix of one of the users for each Internet product, input the behavior interaction matrix of one of the users for each Internet product into the multi-head self-attention layer, obtain the weighted behavior feature vector, input the weighted behavior feature vector into the fully connected layer, and obtain the behavior preference classification result of one of the users for each Internet product.
[0103] In a specific embodiment, step S3 specifically includes:
[0104] The product feature vector and user feature vector are input into the concatenation layer, and are first concatenated according to different interaction behaviors to obtain the interaction behavior vector corresponding to the different interaction behaviors of the k-th user on the i-th Internet product. The calculation formula is as follows:
[0105] ;
[0106] in, =1,2,3, =1 indicates click behavior, =2 indicates collection behavior, =3 indicates additional purchase behavior, represents the product feature vector of the ith Internet product. The product feature vector is a vector composed of the quantified values of product features. Product features include product type, price, warranty period, product status, and sales status. represents the user feature vector of the kth user. The user feature vector is a vector composed of the quantized values of user features. User features include age, gender, occupation, marital status, and number of children. Indicates that and Stitch together according to the same interactive behavior;
[0107] Combine the interaction behavior vectors of each interaction behavior generated by the k-th user on the i-th Internet product to obtain the behavior interaction matrix of the k-th user on the i-th Internet product , as shown below:
[0108] ;
[0109] The behavior interaction matrix of the kth user for the i-th Internet product is input into the multi-head self-attention layer, and calculations are performed on each self-attention head to obtain the output of each self-attention head. The calculation formula is as follows:
[0110] ;
[0111] ;
[0112] ;
[0113] ;
[0114] in, represents the output of the ηth self-attention head in the multi-head self-attention layer, η=1,2,…,o, Represents the function corresponding to the self-attention mechanism, represents the behavioral interaction matrix of the kth user on the i-th Internet product, denote the learnable matrices corresponding to the nth query vector, the nth key vector, and the nth value vector, respectively, , and represents the nth query vector, the nth key vector and the nth value vector, and D represents The dimension of , T represents transpose;
[0115] The outputs of each self-attention head are concatenated to obtain the weighted behavioral feature vector of the kth user for the i-th Internet product , the formula is as follows:
[0116] ;
[0117] The weighted behavioral feature vector of the k-th user for the i-th Internet product is input into the fully connected layer to obtain the category probability matrix. The calculation formula is as follows:
[0118] ;
[0119] in, represents the class probability vector, represents the weight matrix, express The activation function, b represents the bias, each element in the category probability vector represents the probability of the k-th user generating each interactive behavior on the ith Internet product, and the interactive behavior corresponding to the maximum value of all elements in the category probability vector is taken as the behavioral preference classification result of the k-th user on the ith Internet product.
[0120] Specifically, by analyzing the interactive behavior information between Internet insurance users and products, the user feature vector corresponding to one of the users in the user group and the product feature vector corresponding to each Internet product are spliced according to the corresponding interactive behavior to obtain the interactive behavior vector of one of the users with different interactive behaviors on one of the Internet products. The information between users and Internet products is combined to provide richer feature representation. The interactive behavior vectors of all interactive behaviors of one of the users on one of the Internet products are spliced to obtain the behavioral interaction matrix of one of the users on one of the Internet products. The splicing operation retains the original feature information and avoids the loss of information. The operation is simple and intuitive, easy to understand and implement. The behavioral interaction matrix is used as the input of the multi-head self-attention layer to learn the user's behavioral preferences. The behavioral interaction matrix of one of the users on one of the Internet products is input into the multi-head self-attention layer, and the behavioral interaction matrix is weightedly represented using the self-attention weight, so as to learn the user's behavioral preferences for different products. Through the self-attention mechanism, different attention weights can be dynamically learned to assign different interactive behavior vectors to capture the complex relationship and changes of user behavioral preferences. The weighted behavior feature vector output by the multi-head self-attention layer is used as the input of the fully connected layer. The fully connected layer is linearly transformed through multiple neurons and mapped to the output space in combination with the softmax activation function to perform the classification task. Each neuron in the fully connected layer is connected to all neurons in the previous layer. After the transformation of the fully connected layer, a category probability matrix can be obtained. For each insurance product, the category probability matrix contains the probability prediction of the user's three interactive behaviors, namely the probability of clicking, collecting and adding to the purchase. The interactive behavior corresponding to the maximum probability is the behavior preference classification result. These behavior preference classification results will serve as the basis for the subsequent generation of insurance product bundles. The user behavior preference classification model will be output. Through the self-attention mechanism, the user behavior preference classification model can adaptively focus on the insurance user behavior information that is more important to the classification task, thereby improving the accuracy of user behavior classification.
[0121] S4, based on the behavioral interaction order set of one of the users in the user group and the behavioral preference classification results of all Internet products, determine the starting point for generating the bundle combination, based on the product order map and the user's behavioral preference classification results, use the beam search algorithm to search for extended nodes from the starting point to form a bundle combination, obtain a bundle combination list, screen the bundle combinations in the bundle combination list, and obtain a recommendation list.
[0122] In a specific embodiment, determining the starting point for bundling combination generation based on the behavior interaction order set of one of the users in the user group and the behavior preference classification results of all Internet products specifically includes:
[0123] The behavior preference classification results of the kth user for all Internet products are classified according to different interactive behaviors, and under each interactive behavior, the probability of the kth user generating this interactive behavior for each Internet product is sorted from large to small, and the top h Internet products are selected to form a product set corresponding to each interactive behavior. , =1,2,3, =1 indicates click behavior, =2 indicates collection behavior, =3 indicates additional purchase behavior;
[0124] From the product collection corresponding to the add-to-cart behavior Determine the starting point of bundle combination generation;
[0125] In response to determining that there is an Internet product that generates an add-to-purchase behavior in the behavior interaction sequence set of the kth user, determine the Internet product that most recently generated the add-to-purchase behavior from the behavior interaction sequence set of the kth user according to the most recent add-to-purchase time of the kth user in the time dimension, and calculate the Internet product that most recently generated the add-to-purchase behavior and the product set corresponding to the add-to-purchase behavior The similarity of Internet products in the product set corresponding to the add-to-cart behavior The Internet product with the highest similarity to the Internet product that generated the most recent purchase behavior is selected as the starting point for generating the bundle combination for the kth user;
[0126] In response to determining that there is no Internet product that generates the add-to-cart behavior in the behavior interaction sequence set of the kth user, the Internet product that most recently generated the collection behavior is determined from the behavior interaction sequence set of the kth user in the time dimension according to the most recent collection time of the kth user, and the Internet product that most recently generated the collection behavior and the product set corresponding to the collection behavior are calculated. The similarity of Internet products in the product set corresponding to the collection behavior The Internet product with the highest similarity to the Internet product that generated the most recent collection behavior is selected as the starting point for generating the bundle combination of the kth user;
[0127] In response to determining that there is no Internet product that generates the add-to-cart and favorite behaviors in the behavior interaction sequence set of the kth user, the Internet product that most recently generated the click behavior is determined from the behavior interaction sequence set of the kth user in the time dimension according to the most recent click time of the kth user, and the Internet product that most recently generated the click behavior and the product set corresponding to the click behavior are calculated. The similarity of Internet products in the set of products corresponding to the click behavior The Internet product with the highest similarity to the Internet product that generated the most recent click behavior is selected as the starting point for generating the bundle combination of the kth user.
[0128] In a specific embodiment, based on the product sequence diagram and the user's behavior preference classification results, a beam search algorithm is used to search for expansion nodes from a starting point to form a bundle combination, obtain a bundle combination list, and screen the bundle combinations in the bundle combination list to obtain a recommendation list, which specifically includes:
[0129] S41, initialization, starting from the starting point generated by the bundle combination of the kth user, and taking the starting point as the current candidate sequence;
[0130] S42, expanding the current candidate sequence, selecting each adjacent node that can be reached from the last node of the current candidate sequence from the product sequence general graph as an expansion node;
[0131] S43, calculating the jump probability between the last node of the current candidate sequence and each extended node, as shown below:
[0132] ;
[0133] ;
[0134] in, Indicates the last node of the current candidate sequence With extension nodes The jump probability between represents the path from one node to another node. Indicates the last node in the product sequence graph from the current candidate sequence Departure to the expansion node The weight of the directed edge of Indicates the last node of the current candidate sequence in the product sequence graph under different interaction behaviors Departure to the expansion node The weight of the directed edge of Represents a collection of expansion nodes;
[0135] S44, selecting beam width parameter nodes with the highest jump probability and not existing in the existing candidate sequence from the extended nodes, and forming the next candidate sequence with the current candidate sequence respectively;
[0136] S45, taking the next candidate sequence as the current candidate sequence and repeating steps S42-S44 until the next candidate sequence reaches the end condition, taking each candidate sequence as a bundle combination to form a bundle combination list for the kth user ;
[0137] Measure the importance of each Internet product in the bundled combination list of the kth user and obtain the frequency of each Internet product in the bundled combination list , as shown below:
[0138] ;
[0139] in, Indicates the eth Internet product List of bundle combinations for the kth user The frequency of occurrence in Indicates the eth Internet product List of bundle combinations for the kth user The number of occurrences in Represents the bundle combination list of the kth user The number of all Internet products in
[0140] According to frequency Sort in descending order, select the top U% of Internet products as important products, and put them in the bundle combination list of the kth user Select the bundles that contain at least one important product to form the first list ;
[0141] The first list is calculated using the following formula The similarity of two different bundle combinations in:
[0142] ;
[0143] ;
[0144] ;
[0145] in, and Represents two different bundle combinations of the same user. and Represents two bundle combinations and The product feature vector corresponding to the Internet products in Represents two bundle combinations and The similarities between Internet products in Represents the similarity of two different bundles for the same user;
[0146] In the first list The bundle combinations whose similarity between two different bundle combinations is lower than the similarity threshold are selected to form the second list ;
[0147] Calculate the second list The δth bundle combination in The accumulated points , as shown below:
[0148] ;
[0149] ;
[0150] in, represents the kth user, Denotes the δth bundle combination Any two Internet products and The weight between represents the δth bundle combination under different interaction behaviors Middle slave node Departure to the node The weights of the directed edges of ;
[0151] For the second list All bundles in The accumulated points are sorted from large to small, and the top T bundle combinations are selected to form a recommendation list .
[0152] Specifically, all Internet products are ranked according to the probability prediction of each interactive behavior. For each interactive behavior, the top h Internet products are selected according to the probability ranking to form three product sets:
[0153] Click Collection : The set of products that a user is most likely to click on.
[0154] Collection : The set of products that users are most likely to collect.
[0155] Add to cart : The set of products that a user is most likely to add to their shopping cart.
[0156] The result of the above classification is to obtain a set of insurance products with three interactive behaviors. The insurance product set often involves multiple insurance products, so it is necessary to select the insurance products from the add-on collection. The starting point of bundling combination generation is established in the time dimension. First, according to the time of the user's most recent purchase, the insurance product corresponding to the user's most recent purchase behavior is determined in the user's behavior interaction sequence set. The similarity measurement is performed on the insurance products related to the user, and the product with the highest similarity to the insurance product corresponding to the user's most recent purchase behavior is selected as the starting point. In the embodiment of the present application, the cosine similarity between the product feature vectors corresponding to two Internet products is used for similarity measurement. In other embodiments, other similarity measurement methods can be selected. If there is no insurance product with purchase behavior in the user's behavior interaction order set, the insurance product corresponding to the user's most recent collection behavior is determined in the user's behavior interaction order set according to the user's most recent collection time, and the collection set is sorted. The similarity of the insurance products in the user's most recent collection behavior is measured, and the product with the highest similarity to the insurance product corresponding to the user's most recent collection behavior is selected as the starting point. If there is no insurance product with added purchase behavior or collection behavior in the user's behavior interaction order set, the insurance product corresponding to the user's most recent click behavior is determined in the user's behavior interaction order set according to the user's most recent click time, and the click set is The similarity of related insurance products is measured, and the product with the highest similarity to the insurance product corresponding to the user's most recent click behavior is selected as the starting point. The priority order of selecting the starting point is add to cart, favorite, and click. This way, the starting point is more in line with the user's current behavioral preferences. The starting point is more likely to be related to the insurance user's current interests, which improves the personalization of the bundled combination. This can also take into account the current timeliness needs of insurance users and better reflect the user's recent purchasing intentions.
[0157] Based on the product sequence diagram of the interaction behavior of the low-purchase user group and the classification results of the behavior preferences of low-purchase users obtained above, the two are used as inputs for generating insurance product bundles. The product sequence diagram of the interaction behavior contains rich information about the different behaviors of insurance users. The weights between insurance products in the diagram also measure the importance of the two products in the interaction information of insurance user behavior. The product sequence diagram based on the interaction behavior can effectively provide the association relationship between insurance products required in the insurance product bundle.
[0158] In order to customize a personalized insurance product combination for a low-purchase user group, the embodiment of the present application adopts an efficient search strategy, the beam search algorithm, which can deeply mine user behavior data, identify potential product combinations, and take into account user behavior preferences for different insurance products.
[0159] The beam width parameter is a key setting in the beam search algorithm, which determines the number of candidate solutions considered simultaneously during the search process. By adjusting the beam width, a balance can be found between the breadth and depth of the search. A larger beam width can increase the possibility of finding the optimal solution, but it will also increase the computational cost. Therefore, this application needs to reasonably set this parameter based on the actual computing resources and time constraints.
[0160] Minimum Generation Length is another important constraint that specifies the minimum number of insurance products that must be included in the candidate solution before the search is terminated. This parameter ensures that the generated product bundles have a certain complexity and practicality, avoiding the recommendation of overly simple or unattractive bundles.
[0161] In each step of the search on the product order graph, the beam search algorithm generates several expansion nodes of insurance products and selects the most likely expansion node as the candidate insurance product for the next step. In this process, we should pay attention to the weights between insurance products in the product order graph.
[0162] The beam search algorithm starts from the starting point, selects the most likely expansion node according to the jump probability formula, that is, expands within the maximum probability calculated by the jump formula, and gradually builds potential product bundle combinations until the minimum generation length is met.
[0163] The search process of the beam search algorithm is as follows:
[0164] (1) Initialization: The search of the beam search algorithm starts from the starting point obtained previously, taking it as the first candidate sequence, and performing the following steps with the first candidate sequence as the current sequence.
[0165] (2) Expand the current candidate sequence. For each node in the current candidate sequence, consider all possible expansion nodes (i.e., other insurance products) in the product sequence graph starting from the last node in the current candidate sequence, and form all possible edge paths.
[0166] (3) Probability calculation: For each possible expansion node, calculate the probability of the path passing through the edge.
[0167] (4) Selection and pruning. For all the extended nodes of each candidate sequence, select the ones with the highest probability according to the probability, that is, the previously set beam width parameter extended nodes. That is, from all these extended nodes, select the previously set beam width parameter extended nodes with the highest probability again to combine with the current candidate sequence to form the next candidate sequence.
[0168] (5) Repeat steps 2-4 above with the next candidate sequence as the current candidate sequence until a candidate sequence reaches the minimum generation length requirement, that is, meets the set minimum number of insurance products.
[0169] Since the essence of the product sequence graph is a weighted directed graph, its complexity is difficult to predict. Therefore, during the generation process, an insurance product may appear repeatedly in the same bundle combination. In order to avoid this situation, after calculating the possible candidate products, it is determined whether they are already in the existing bundle combination. If not, they are added to the candidate sequence.
[0170] Through the above steps, a bundle combination list containing several bundle combinations is obtained. The bundle combination list is calculated based on the user behavior data and the degree of association between products. However, the initially generated bundle combination list may contain bundle combinations with high similarity, which may make users feel that their choices are limited and lack personalization, thus resulting in a decline in user experience. Therefore, before delivering it to users, it is necessary to consider how to reduce the number of bundle combinations with too high similarity, that is, to discard bundle combinations with too high similarity and relatively low cumulative points ranking. This ensures that the recommended list of insurance products generated in the end is more diverse and representative.
[0171] It is necessary to measure the importance of insurance products in the bundle list by calculating the frequency of insurance products in the entire bundle list to understand the importance of each insurance product in the entire bundle list. This helps to identify which products are core to user preferences and which products may appear repeatedly in multiple bundles, thus affecting diversity.
[0172] After that, similarity is calculated. In the embodiment of the present application, bulldozing distance is used to calculate the similarity of each bundle. Bulldozing distance is a method for measuring the difference between two probability distributions, which can be applied to evaluate the similarity between bundles. By calculating the bulldozing distance between each combination and all other combinations, the difference between them can be quantified.
[0173] Then, those bundles with relatively low similarity but high cumulative scores, which also include important products in user preferences, are screened out and recommended to users.
[0174] For the results of the Internet insurance product bundles obtained after importance and similarity screening, the cumulative score is proposed as an evaluation indicator to measure the value of the bundles included in the bundle list. The data basis of the evaluation indicator comes from the weights between insurance products in the product order diagram of the interactive behavior of low-purchase users. By calculating the cumulative score of each bundle, all generated bundles can be ranked. Bundles with high cumulative scores may be more popular with users, so they can be considered as the first choice for recommendation.
[0175] Through the above calculations, the embodiment of the present application obtains the importance of each insurance product, the cumulative score of the bundle combination, and the similarity between two bundle combinations. Now, according to the above, we will filter. First, we set the criteria for important products. According to the importance of each insurance product in the entire bundle combination list, we will calculate the similarity between two bundle combinations. In descending order, in one embodiment, the top 10% of Internet products can be selected as important products, and in other embodiments, it can be selected to mark the important products according to the above standards, and select the bundled combination containing at least one important product to form the first list ; Secondly, set the similarity threshold for the first list Screen and obtain the bundle combinations below the similarity threshold to form the second list ; Again, sort the second list according to the accumulated points The bundle combinations in the list are sorted in descending order; finally, the first T combinations are selected from the sorted list to form a recommendation list , the recommended list Provided to users.
[0176] Through the above methods, the personalization level of insurance product bundles can be significantly improved, while enriching the user's choice range. Users can therefore choose from a series of carefully selected and distinctive insurance product combinations, which not only meets personalized needs but also improves the overall purchasing experience.
[0177] The above steps S1-S4 do not necessarily represent the order between the steps, but are represented by step symbols, and the order between the steps can be adjusted.
[0178] Further references Figure 4 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a combined recommendation device of a self-attention mechanism for low-purchase users. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0179] The present application embodiment provides a combined recommendation device of a self-attention mechanism for low-purchase users, including:
[0180] The product order graph generation module 1 is configured to obtain the behavior interaction information of each user in the user group and obtain a behavior interaction order set according to the time when the interaction behavior occurs, classify and summarize the behavior interaction order set according to different interaction behaviors, obtain a product order subset of each user under different interaction behaviors, and the interaction behaviors include click, favorite and add to cart, respectively superimpose the product order subsets of all users under different interaction behaviors to obtain a product order graph generated by the user group under each interaction behavior, and merge the product order graphs generated by the user group under all interaction behaviors to obtain a product order graph;
[0181] A model building module 2 is configured to build and train a user behavior preference classification model to obtain a trained user behavior preference classification model, wherein the user behavior preference classification model includes a concatenation layer, a multi-head self-attention layer, and a fully connected layer connected in sequence;
[0182] The behavior preference prediction module 3 is configured to obtain a user feature vector corresponding to one of the users in the user group and a product feature vector corresponding to each Internet product and input them into a trained user behavior preference classification model, input the user feature vector and the product feature vector into a concatenation layer, obtain a behavior interaction matrix of one of the users for each Internet product, input the behavior interaction matrix of one of the users for each Internet product into a multi-head self-attention layer, obtain a weighted behavior feature vector, input the weighted behavior feature vector into a fully connected layer, and obtain a behavior preference classification result of one of the users for each Internet product;
[0183] The screening module 4 is configured to determine the starting point for generating a bundle combination based on the behavioral interaction order set of one of the users in the user group and the behavioral preference classification results of all Internet products, and to use a beam search algorithm to search for extended nodes from the starting point to form a bundle combination based on the product order overall graph and the user's behavioral preference classification results, to obtain a bundle combination list, and to screen the bundle combinations in the bundle combination list to obtain a recommendation list.
[0184] Figure 5 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Figure 5 As shown, the electronic device of this embodiment includes: a processor 501 and a memory 502; wherein the memory 502 is used to store computer-executable instructions; the processor 501 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description in the above method embodiment.
[0185] Optionally, the memory 502 may be independent or integrated with the processor 501 .
[0186] When the memory 502 is independently provided, the electronic device further includes a bus 503 for connecting the memory 502 and the processor 501 .
[0187] The embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 501 executes the computer execution instructions, the above method is implemented.
[0188] The embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 501, the above method is implemented.
[0189] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.
[0190] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.
[0191] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit formed by the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0192] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 501 to perform some steps of the methods of various embodiments of the present application.
[0193] It should be understood that the processor 501 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor 501 may be any conventional processor 501, etc. The steps of the method disclosed in the invention may be directly embodied in the execution of the hardware processor 501, or may be executed by a combination of hardware and software modules in the processor 501.
[0194] The memory 502 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.
[0195] The bus 503 may be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 503 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 503 in the drawings of the present application is not limited to only one bus 503 or one type of bus 503.
[0196] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.
[0197] An exemplary storage medium is coupled to the processor 501, so that the processor 501 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 501. The processor 501 and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor 501 and the storage medium can also exist as discrete components in an electronic device or a main control device.
[0198] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A combined recommendation method of self-attention mechanism for low-purchase users, characterized in that: The following steps are involved: Obtaining the behavior interaction information of each user in the user group and obtaining a behavior interaction order set according to the time when the interaction behavior occurs, classifying and summarizing the behavior interaction order set according to different interaction behaviors to obtain a product order subset of each user under different interaction behaviors, wherein the interaction behaviors include click, favorite, and add to cart, superimposing the product order subsets of all users under different interaction behaviors respectively to obtain a product order graph generated by the user group under each interaction behavior, and merging the product order graphs generated by the user group under all interaction behaviors to obtain a product order overall graph; Constructing and training a user behavior preference classification model to obtain a trained user behavior preference classification model, wherein the user behavior preference classification model includes a concatenation layer, a multi-head self-attention layer, and a fully connected layer connected in sequence; Obtain a user feature vector corresponding to one of the users in the user group and a product feature vector corresponding to each Internet product and input them into a trained user behavior preference classification model, input the user feature vector and the product feature vector into a concatenation layer, obtain a behavior interaction matrix of one of the users for each Internet product, input the behavior interaction matrix of one of the users for each Internet product into the multi-head self-attention layer, obtain a weighted behavior feature vector, input the weighted behavior feature vector into the fully connected layer, and obtain a behavior preference classification result of one of the users for each Internet product; Based on the behavioral interaction order set of one of the users in the user group and the behavioral preference classification results of all Internet products, a starting point for generating a bundle combination is determined; based on the product order overall graph and the user's behavioral preference classification results, a beam search algorithm is used to search for extended nodes from the starting point to form a bundle combination, and a bundle combination list is obtained; the bundle combinations in the bundle combination list are screened to obtain a recommendation list.
2. The combined recommendation method of the self-attention mechanism for low-purchase users according to claim 1, characterized in that: The behavior interaction order set of one of the users in the user group is: , , Represents a subset of product order generated by the click behavior of one of the users in the user group; Represents a subset of product order generated by the collection behavior of one user in the user group; represents a subset of product order generated by the purchase behavior of one of the users in the user group, where the order is determined by the time when the behavior occurs; the product order graph of the user group under different interactive behaviors is represented as , , =1,2,3, =1 indicates click behavior, =2 indicates collection behavior, =3 indicates additional purchase behavior, where: , Represents the node set in the product sequence graph under different interaction behaviors. The i-th Internet product Represents the i-th node , i=1,2,…,n, represents the edge set in the product order graph under different interaction behaviors, , , g = 1, 2, …, m, Represented as slave nodes under different interaction behaviors Departure to the node And the weight is The directed edges of The calculation formula is as follows: ; ; in, represents the kth user in the user group, c represents the total number of users in the user group, Indicates the first Internet product and the jth Internet product The weight corresponding to performing one of the interactive behaviors, for The indicator function of Click on the Internet product ,but ,otherwise , for The indicator function of Click on the Internet product ,but ,otherwise .
3. The combined recommendation method of the self-attention mechanism for low-purchase users according to claim 2, characterized in that: The product sequence graphs generated by the user group under all interaction behaviors are merged to obtain a product sequence master graph, which specifically includes: The node set of the product order graph is the union of the node sets of the product order graphs generated under all interaction behaviors, the edge set of the product order graph is the union of the edge sets of the product order graphs generated under all interaction behaviors, and the weights of the directed edges in the edge set of the product order graph are the sum of the weights of the directed edges in the edge set of the product order graph generated under each interaction behavior.
4. The combined recommendation method of self-attention mechanism for low-purchase users according to claim 1, characterized in that: The user feature vector corresponding to one of the users in the user group and the product feature vector corresponding to each Internet product are obtained and input into the trained user behavior preference classification model. The user feature vector and the product feature vector are input into the concatenation layer to obtain the behavior interaction matrix of one of the users for each Internet product. The behavior interaction matrix of one of the users for each Internet product is input into the multi-head self-attention layer to obtain a weighted behavior feature vector. The weighted behavior feature vector is input into the fully connected layer to obtain the behavior preference classification result of one of the users for each Internet product, which specifically includes: The product feature vector and user feature vector are input into the splicing layer, and are first spliced according to different interactive behaviors to obtain the interactive behavior vector corresponding to the different interactive behaviors of the k-th user on the i-th Internet product. The calculation formula is as follows: ; in, =1,2,3, =1 indicates click behavior, =2 indicates collection behavior, =3 indicates additional purchase behavior, represents the product feature vector of the ith Internet product, where the product feature vector is a vector composed of quantized values of product features, including product type, price, warranty period, product status, and sales status. represents the user feature vector of the kth user, the user feature vector being a vector composed of quantized values of user features, the user features including age, gender, occupation, marital status, and number of children, Indicates that and Stitch together according to the same interactive behavior; Combine the interaction behavior vectors of each interaction behavior generated by the k-th user on the i-th Internet product to obtain the behavior interaction matrix of the k-th user on the i-th Internet product , as shown below: ; The behavior interaction matrix of the k-th user on the i-th Internet product is input into the multi-head self-attention layer, and calculations are performed on each self-attention head to obtain the output of each self-attention head. The calculation formula is as follows: ; ; ; ; in, represents the output of the nth self-attention head in the multi-head self-attention layer, n=1,2,…,o, Represents the function corresponding to the self-attention mechanism, represents the behavioral interaction matrix of the kth user on the i-th Internet product, denote the learnable matrices corresponding to the nth query vector, the nth key vector, and the nth value vector, respectively, , and represents the nth query vector, the nth key vector and the nth value vector, and D represents The dimension of , T represents transpose; The outputs of each self-attention head are concatenated to obtain the weighted behavioral feature vector of the kth user for the i-th Internet product , the formula is as follows: ; The weighted behavior feature vector of the k-th user on the i-th Internet product is input into the fully connected layer to obtain the category probability matrix, which is calculated as follows: ; in, represents the class probability vector, represents the weight matrix, express The activation function, b represents the bias, each element in the category probability vector represents the probability of the k-th user generating each interactive behavior on the ith Internet product, and the interactive behavior corresponding to the maximum value of all elements in the category probability vector is taken as the behavioral preference classification result of the k-th user on the ith Internet product.
5. The combined recommendation method of self-attention mechanism for low-purchase users according to claim 1, characterized in that: The starting point for bundling combination generation is determined based on the behavioral interaction sequence set of one of the users in the user group and the behavioral preference classification results of all Internet products, including: The behavior preference classification results of the kth user for all Internet products are classified according to different interactive behaviors, and under each interactive behavior, the probability of the kth user generating this interactive behavior for each Internet product is sorted from large to small, and the top h Internet products are selected to form a product set corresponding to each interactive behavior. , =1,2,3, =1 indicates click behavior, =2 indicates collection behavior, =3 indicates additional purchase behavior; From the product collection corresponding to the add-to-cart behavior Determining a starting point for generating the bundle combination; In response to determining that there is an Internet product that generates an additional purchase behavior in the behavior interaction sequence set of the kth user, determine the Internet product that most recently generated the additional purchase behavior from the behavior interaction sequence set of the kth user according to the most recent additional purchase time of the kth user in the time dimension, and calculate the Internet product that most recently generated the additional purchase behavior and the product set corresponding to the additional purchase behavior The similarity of Internet products in the product set corresponding to the add-to-cart behavior The Internet product with the highest similarity to the Internet product that generated the most recent purchase behavior is selected as the starting point for generating the bundle combination for the kth user; In response to determining that there is no Internet product that generates the add-to-cart behavior in the behavior interaction order set of the k-th user, the Internet product that most recently generated the collection behavior is determined from the behavior interaction order set of the k-th user in the time dimension according to the most recent collection time of the k-th user, and the Internet product that most recently generated the collection behavior and the product set corresponding to the collection behavior are calculated. The similarity of Internet products in the product set corresponding to the collection behavior The Internet product with the highest similarity to the Internet product that generated the most recent collection behavior is selected as the starting point for generating the bundle combination of the kth user; In response to determining that there is no Internet product that generates the add-to-cart and favorite behaviors in the behavior interaction sequence set of the k-th user, the Internet product that most recently generated the click behavior is determined from the behavior interaction sequence set of the k-th user in the time dimension according to the most recent click time of the k-th user, and the Internet product that most recently generated the click behavior and the product set corresponding to the click behavior are calculated. The similarity of Internet products in the set of products corresponding to the click behavior The Internet product with the highest similarity to the Internet product that generated the most recent click behavior is selected as the starting point for generating the bundle combination of the kth user.
6. The combined recommendation method of the self-attention mechanism for low-purchase users according to claim 5, characterized in that: Based on the product sequence diagram and the user's behavior preference classification results, a beam search algorithm is used to search for expansion nodes from the starting point to form a bundle combination, obtain a bundle combination list, and filter the bundle combinations in the bundle combination list to obtain a recommendation list, specifically including: S41, initialization, starting from the starting point generated by the bundled combination of the kth user, and taking the starting point as the current candidate sequence; S42, expanding the current candidate sequence, selecting from the product sequence graph each adjacent node that can be reached from the last node of the current candidate sequence as the expansion node; S43, calculating the jump probability between the last node of the current candidate sequence and each extended node, as shown below: ; ; in, Indicates the last node of the current candidate sequence With extension nodes The jump probability between represents the path from one node to another node. Indicates the last node of the current candidate sequence in the product sequence graph Departure to the expansion node The weight of the directed edge of Indicates the last node of the current candidate sequence in the product sequence graph under different interaction behaviors Departure to the expansion node The weight of the directed edge of Represents a collection of expansion nodes; S44, selecting beam width parameter nodes with the highest jump probability and not existing in the existing candidate sequence from the extended nodes, and forming the next candidate sequence with the current candidate sequence respectively; S45, taking the next candidate sequence as the current candidate sequence and repeating steps S42-S44 until the next candidate sequence reaches the end condition, taking each candidate sequence as a bundle combination to form a bundle combination list for the kth user ; The importance of each Internet product in the bundled combination list of the kth user is measured to obtain the frequency of each Internet product in the bundled combination list. , as shown below: ; in, Indicates the eth Internet product List of bundle combinations for the kth user The frequency of occurrence in Indicates the eth Internet product List of bundle combinations for the kth user The number of occurrences in Represents the bundle combination list of the kth user The number of all Internet products in According to the frequency Sort in descending order, select the top U% of Internet products as important products, and put them in the bundle combination list of the kth user Select the bundles that contain at least one important product to form the first list ; The first list is calculated using the following formula The similarity of two different bundle combinations in : ; ; ; in, and Represents two different bundle combinations of the same user. and Represents two bundle combinations and The product feature vector corresponding to the Internet products in Represents two bundle combinations and The similarities between Internet products in Represents the similarity of two different bundles of the same user; In the first list The bundle combinations whose similarity between two different bundle combinations is lower than the similarity threshold are selected to form the second list ; Calculate the second list The δth bundle combination in The accumulated points , as shown below: ; ; in, represents the kth user, Denotes the δth bundle combination Any two Internet products and The weight between represents the δth bundle combination under different interaction behaviors Middle slave node Departure to the node The weights of the directed edges of ; For the second list All bundles in The accumulated points are sorted from large to small, and the top T bundle combinations are selected to form a recommendation list .
7. A combined recommendation device based on a self-attention mechanism for low-purchase users, characterized in that: include: A product order map generation module is configured to obtain the behavior interaction information of each user in the user group and obtain a behavior interaction order set according to the time when the interaction behavior occurs, classify and summarize the behavior interaction order set according to different interaction behaviors, obtain a product order subset of each user under different interaction behaviors, and the interaction behaviors include click, favorite and add to purchase, respectively superimpose the product order subsets of all users under different interaction behaviors to obtain a product order map generated by the user group under each interaction behavior, and merge the product order maps generated by the user group under all interaction behaviors to obtain a product order map; A model building module is configured to build and train a user behavior preference classification model to obtain a trained user behavior preference classification model, wherein the user behavior preference classification model includes a concatenation layer, a multi-head self-attention layer, and a fully connected layer connected in sequence; The behavior preference prediction module is configured to obtain a user feature vector corresponding to one of the users in the user group and a product feature vector corresponding to each Internet product and input them into a trained user behavior preference classification model, input the user feature vector and the product feature vector into a concatenation layer, obtain a behavior interaction matrix of one of the users for each Internet product, input the behavior interaction matrix of one of the users for each Internet product into the multi-head self-attention layer, obtain a weighted behavior feature vector, input the weighted behavior feature vector into the fully connected layer, and obtain a behavior preference classification result of one of the users for each Internet product; The screening module is configured to determine the starting point for generating a bundle combination based on a set of behavioral interaction orders of one of the users in the user group and a behavioral preference classification result of all Internet products, and to use a beam search algorithm to search for extended nodes from the starting point to form a bundle combination based on the product order master map and the user's behavioral preference classification result, to obtain a bundle combination list, and to screen the bundle combinations in the bundle combination list to obtain a recommendation list.
8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Content recommendation method and device, electronic equipment and storage medium
CN114764445A
Project recommendation method and related equipment thereof
CN115292583A
User multi-behavior recommendation method based on graph neural network and meta learning
CN117951391A
Internet insurance product binding method, device, equipment and medium
CN118313937A
Automatic recommendation of analysis for dataset
WO2021108053A1
