A product recommendation method, device, storage medium and system

Abstract

The present application provides a product recommendation method, device, storage medium, and system. The method includes: obtaining multiple feature data to be calculated corresponding to multiple candidate products; inputting the multiple feature data to be calculated into a target recommendation model, and obtaining click probability values ​​and purchase probability values ​​for each of the multiple candidate products output by the target recommendation model; the target recommendation model is trained based on at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample; based on the click probability values ​​and purchase probability values ​​of the multiple candidate products, the recommendation scores of the multiple candidate products are calculated; and based on the recommendation scores of the multiple candidate products, the products are recommended to the user to be recommended. In this way, the target recommendation model is trained based on high-quality positive samples, low-quality positive samples, and negative samples, which enables the target recommendation model to learn more in-depth features and improve the accuracy of product recommendations.

Description

Technical Field

[0001] The present application relates to the technical field of product recommendation, and in particular to a product recommendation method, device, storage medium, and system. Background Art

[0002] With the development of big data and artificial intelligence technologies, more and more users are enjoying online shopping. To attract more traffic and boost user activity, major e-commerce platforms need to design personalized product recommendations to increase sales. Currently, existing product recommendation algorithms typically extract multiple product features based on the recommendation process, then assign linearly weighted coefficients to each product to score it. The algorithms then rank the products based on the scores and recommend them to users.

[0003] However, for existing product recommendation algorithms, the multiple features extracted are manually designed and mostly rely on business experience, which has certain limitations. In addition, the feature information learned by the linear weighted model is relatively superficial, and its ability to sort products also has certain limitations. Summary of the Invention

[0004] The present application provides a product recommendation method, device, storage medium and system, which improves the accuracy of product recommendations by introducing the segmentation logic of high-quality positive samples, low-quality positive samples and negative samples.

[0005] The technical solution of this application is achieved as follows:

[0006] In a first aspect, an embodiment of the present application provides a product recommendation method, the method comprising:

[0007] Obtain multiple feature data to be calculated corresponding to multiple candidate products;

[0008] Inputting the plurality of feature data to be calculated into a target recommendation model, and obtaining click probability values ​​and purchase probability values ​​of the plurality of candidate products output by the target recommendation model; wherein the target recommendation model is trained based on at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample;

[0009] Calculating a recommendation score for each of the plurality of candidate products based on the click probability value and the purchase probability value of each of the plurality of candidate products;

[0010] Recommend products to the user to be recommended based on the recommendation scores of the multiple candidate products.

[0011] In a second aspect, an embodiment of the present application provides a product recommendation device, which includes a data unit, a calculation unit, a determination unit, and a recommendation unit; wherein,

[0012] A data unit configured to obtain a plurality of feature data to be calculated corresponding to a plurality of candidate products;

[0013] a computing unit configured to input the plurality of feature data to be calculated into a target recommendation model, and obtain a click probability value and a purchase probability value of each of the plurality of candidate products output by the target recommendation model; wherein the target recommendation model is trained based on at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample;

[0014] a determining unit configured to calculate a recommendation score for each of the plurality of candidate products based on the click probability value and the purchase probability value of each of the plurality of candidate products;

[0015] The recommendation unit is configured to recommend products to the user to be recommended based on the recommendation scores of the multiple candidate products.

[0016] In a third aspect, an embodiment of the present application provides a product recommendation device, which includes a memory and a processor; wherein:

[0017] a memory for storing a computer program capable of running on the processor;

[0018] A processor is configured to execute the steps of the method described in the first aspect when running the computer program.

[0019] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the steps of the method described in the first aspect.

[0020] In a fifth aspect, the present application provides a product recommendation system, which at least includes the product recommendation device as described in the second aspect or the third aspect.

[0021] The embodiments of the present application provide a product recommendation method, device, storage medium and system, which obtain multiple feature data to be calculated corresponding to multiple candidate products; input the multiple feature data to be calculated into a target recommendation model, and obtain the click probability value and purchase probability value of each of the multiple candidate products output by the target recommendation model; wherein the target recommendation model is trained based on at least one high-quality positive sample, at least one low-quality positive sample and at least one negative sample; based on the click probability value and purchase probability value of each of the multiple candidate products, the recommendation score of each of the multiple candidate products is calculated; based on the recommendation score of each of the multiple candidate products, the product is recommended to the user to be recommended. In this way, the target recommendation model is trained by high-quality positive samples, low-quality positive samples and negative samples, which enables the target recommendation model to learn more in-depth features, thereby improving the accuracy of product recommendations and ultimately improving product conversion rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a product recommendation method provided in an embodiment of the present application;

[0023] Figure 2 A flowchart of another product recommendation method provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of an application scenario of a product recommendation method provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of the structure of a product recommendation system provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of a product recommendation device provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of the structure of another product recommendation device provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of the hardware structure of a product recommendation device provided in an embodiment of the present application;

[0029] Figure 8 A schematic diagram of the structure of another product recommendation system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0031] The following are the technical terms and their explanations involved in the embodiments of this application:

[0032] Downstream market: The user group likes low-priced goods, is sensitive to promotions, and is keen on interactive sharing, so it has great potential for growth and development.

[0033] Feature engineering: Systematically design multi-dimensional features such as user behavior and product behavior.

[0034] High-quality and low-quality clicks: Distinguish click behaviors based on multiple factors such as user stay time.

[0035] Multi-task learning: is a field in machine learning. Its goal is to use the useful information contained in multiple learning tasks to help obtain a more accurate learner for each task. In the embodiment of this application, multi-task learning is divided into click tasks (or click calculation tasks) and order tasks (or purchase calculation tasks).

[0036] Evaluation indicators: The basis for evaluating the effectiveness of online ranking models, mainly divided into click-based evaluation indicators and order-based evaluation indicators.

[0037] Express Edition: The Express Edition is the specific business scenario of this application, mainly targeting low-end users.

[0038] With the development of big data and artificial intelligence technologies, more and more users are enjoying online shopping. To attract more traffic and increase user activity, major e-commerce platforms need to design personalized product recommendations to increase purchase probability. Currently, existing product recommendation algorithms generally extract multiple features based on the recommendation process, then assign linearly weighted coefficients to score the products, sort them by score, and recommend them to users.

[0039] However, existing product recommendation algorithms have the following shortcomings: the multiple features extracted based on business context are manually designed, mostly relying on business experience, and have certain limitations. In addition, the feature information learned by the linear weighted model is relatively superficial, and its ability to sort products is also limited. The embodiment of the present application designs a systematic feature engineering, improving the sorting model to multi-task learning, which can better learn the deep-level feature representation of the business and improve the online effect.

[0040] An embodiment of the present application provides a product recommendation method, the basic idea of ​​which is: obtaining multiple feature data to be calculated corresponding to multiple candidate products; inputting the multiple feature data to be calculated into a target recommendation model, and obtaining the click probability value and purchase probability value of each of the multiple candidate products output by the target recommendation model; wherein, the target recommendation model is trained based on at least one high-quality positive sample, at least one low-quality positive sample and at least one negative sample; calculating the recommendation score of each of the multiple candidate products based on the click probability value and purchase probability value of each of the multiple candidate products; and recommending products to users to be recommended based on the recommendation scores of each of the multiple candidate products. In this way, during the model training phase, the preset multi-task model can be trained according to the sample feature data, so that the preset multi-task model can learn more in-depth features and ultimately obtain a more accurate target recommendation model; secondly, in terms of label values, the segmentation logic of high-quality positive samples, low-quality positive samples and negative samples is introduced, and sub-scene division is made for high-quality and low-quality clicks, which further improves the accuracy of the target recommendation model; finally, for the preset multi-task model, four optimization points are introduced (self-attention mechanism, distinction between shared experts and unique experts, gradient boosting tree serial mechanism and multi-level training) to improve the use effect of the target recommendation model, thereby improving the accuracy of product recommendations.

[0041] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0042] In one embodiment of the present application, see Figure 1 , which shows a flow chart of a commodity recommendation method provided by an embodiment of the present application. Figure 1 As shown, the method may include:

[0043] S101: Acquire a plurality of feature data to be calculated corresponding to a plurality of candidate commodities.

[0044] It should be noted that the embodiments of this application provide a product recommendation method for providing personalized product recommendations to users. Here, the products can be physical goods or virtual goods such as videos, novels, applications, news, and movies. Therefore, the product recommendation method provided in the embodiments of this application can be applied to major e-commerce platforms, video and movie streaming websites, and chat and communication software.

[0045] It should be noted that when recommending products to the user to be recommended, multiple feature data to be calculated for multiple candidate products are obtained. In a specific embodiment, each feature data to be calculated is a concatenation of the user characteristics of the user to be recommended and the product characteristics of a candidate product, so that the multiple candidate products and the multiple feature data to be calculated are one-to-one corresponding; in this way, the target recommendation model can be used to calculate the click probability value and purchase probability value of each candidate product for the user to be recommended, so as to make product recommendations.

[0046] S102: Input the plurality of feature data to be calculated into a target recommendation model, and obtain click probability values ​​and purchase probability values ​​of the plurality of candidate products output by the target recommendation model.

[0047] It should be noted that the target recommendation model is pre-trained based on actual usage scenarios. Specifically, the target recommendation model is trained based on at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample. In other words, during the model training phase, the training samples further introduce segmentation logic based on the logic of positive and negative samples. Positive samples are further segmented into high-quality positive samples and low-quality positive samples. This allows the target recommendation model to learn more in-depth features and improve the accuracy of subsequent product recommendations.

[0048] In this way, multiple feature data to be calculated are sequentially input into the target recommendation model, and the target recommendation model outputs click probability values ​​and purchase probability values ​​for each of the multiple candidate products.

[0049] It should be noted that the target recommendation model can include model architectures based on various principles, such as the Multi-Gate Mixture-of-Experts (MMoE) multi-task model, the Deep Factorization Machines (DeepFM) multi-task learning model, the Mixture of Sequential Experts (MoSE) multi-task model, the Mixture-of-Experts (MoE) multi-task model, etc., which are not limited in the embodiments of this application. However, for the sake of convenience, the following content is explained using the MMoE multi-task model as an example.

[0050] Specifically, the MMoE multi-task model includes multiple expert sub-models (abbreviated as Expert) and a gating sub-model (abbreviated as Gate). The multiple expert sub-models each output multiple feature matrices. The gating sub-model performs weighted calculations on the multiple feature matrices according to different tasks, thereby obtaining click probability values ​​and purchase probability values, respectively. For the MMoE multi-task model, the gating sub-model actually includes multiple. For the sake of convenience, the embodiments of this application are collectively referred to as "gating sub-models."

[0051] Taking the target recommendation model as an MMoE multi-task model as an example, in some embodiments, the target recommendation model includes at least one first target expert model, at least one second target expert model, at least one shared target expert model, and a target gating sub-model; inputting the multiple feature data to be calculated into the target recommendation model and obtaining the click probability value and purchase probability value of each of the multiple candidate products output by the target recommendation model may include:

[0052] Calculating the plurality of feature data to be calculated using at least one first target expert model, at least one second target expert model, and at least one shared target expert model, respectively, to obtain at least one first output result, at least one second output result, and at least one shared output result;

[0053] Based on the first output result and the shared output result, obtaining a click probability value of one of the candidate products through the target gating sub-model;

[0054] Based on the second output result and the shared output result, obtaining a purchase probability value of one of the candidate products through the target gating sub-model;

[0055] It should be noted that the embodiment of the present application subdivides multiple preset expert submodels into task-specific experts and shared experts to further better characterize the relevance and differences between different tasks. Therefore, the multiple target expert submodels include at least one first target expert model, at least one second target expert model, and at least one shared target expert submodel. The first target expert model and the second target expert model are used to perform the click probability value task and the purchase probability value task, respectively, while the shared target expert submodel is used in both tasks.

[0056] In a specific embodiment, there are eight target expert sub-models: two first target expert sub-models, two second target expert sub-models, and four shared target expert sub-models. That is, when calculating the click probability value, the output results of the two first target expert sub-models and the four shared target expert sub-models are used; when calculating the purchase probability value, the output results of the two second target expert sub-models and the four shared target expert sub-models are used.

[0057] In this way, by subdividing experts into task-specific experts and common experts, the relevance and distinction of different tasks can be better characterized, thereby improving the accuracy of the target recommendation model.

[0058] In addition to the Expert sharing improvements, the target recommendation model also features three additional optimizations compared to the MMoE multi-task model in related technologies: improved Gate attention, the introduction of GDBT, and multi-level training. Based on these four optimizations, the model calculation process in another specific embodiment is described:

[0059] Assuming that the target recommendation model includes N target expert sub-models and a gating sub-model, inputting the plurality of feature data to be calculated into the target recommendation model and obtaining the click probability value and purchase probability value of each of the plurality of candidate products output by the target recommendation model may include:

[0060] Calculating one of the feature data to be calculated using the first target expert sub-model to obtain a first feature matrix of one of the candidate products;

[0061] Calculate the (i-1)th feature matrix and one of the feature data to be calculated using the i-th target expert sub-model to obtain the i-th feature matrix of one of the candidate products; where i is an integer, i is greater than or equal to 2 and less than or equal to N; N is the number of target expert sub-models, and N is a positive integer;

[0062] According to the N feature matrices of one of the candidate commodities, the target gating sub-model is used to respectively calculate a click probability value and a purchase probability value of one of the candidate commodities.

[0063] It should be noted that, taking the MMoE multi-task model as an example, the trained target recommendation model includes multiple target expert sub-models and target gating sub-models. The multiple target expert sub-models are trained by multiple preset expert sub-models, and the target gating sub-model is trained by the preset gating sub-model.

[0064] Here, the target expert sub-model is used to perform deep feature extraction on the calculated feature data, that is, each target expert sub-model outputs a feature matrix, and the target gating sub-model can perform weighted calculations on the feature matrix output by the target expert sub-model according to different tasks (click calculation tasks and purchase calculation tasks), thereby calculating the click probability value and the purchase probability value.

[0065] That is to say, the calculation of the multiple feature data to be calculated by the target recommendation model can be divided into two steps: (1) using multiple target expert sub-models to calculate the feature data to be calculated respectively to obtain multiple feature matrices; (2) based on the multiple feature matrices, using the target gating sub-model to calculate the click probability value and purchase probability value of one of the candidate products respectively.

[0066] It should be noted that, based on the foregoing, the present embodiment utilizes a gradient boosting tree serial mechanism to optimize multiple preset expert sub-models, allowing them to be linked in series. Accordingly, multiple target expert sub-models also utilize a serial mechanism. That is, in the present embodiment, only the first target expert sub-model receives the feature data to be calculated as input; subsequent target expert sub-models receive the feature data to be calculated concatenated with the output of the previous target expert sub-model.

[0067] The output of the expert sub-model is actually a feature matrix. Therefore, in some embodiments, when N is greater than or equal to 3, the multiple target expert sub-models include at least one first expert model, at least one second expert model, and at least one shared expert model; and the calculation of the click probability and purchase probability of one of the candidate products using the target gating sub-model based on the multiple feature matrices of the candidate product may include:

[0068] Determining a first matrix set and a second matrix set from the N feature matrices of one of the candidate products; wherein the first matrix set includes the feature matrix calculated by the at least one first expert model and the feature matrix calculated by the at least one shared expert model, and the second matrix set includes the feature matrix calculated by the at least one second expert model and the feature matrix calculated by the at least one shared expert model;

[0069] Calculating the first matrix set using the target gating sub-model to obtain a click probability value of one of the candidate products;

[0070] The second matrix set is calculated using the target gating sub-model to obtain a purchase probability value of one of the candidate products.

[0071] It should be noted that, based on the foregoing, the embodiments of the present application further subdivide the multiple preset expert sub-models into task-specific experts and shared experts to better characterize the relevance and differences between different tasks. Therefore, the multiple expert sub-models include at least one first expert model, at least one second expert model, and at least one shared expert model. The first and second expert models are used to perform the click probability value task and the purchase probability value task, respectively, while the shared expert model is used in both tasks.

[0072] Furthermore, in some embodiments, calculating the first matrix set using the target gating sub-model to obtain a click probability value of one of the candidate products may include:

[0073] Performing self-attention calculation on the first matrix set according to a plurality of preset first query vectors, obtaining a first self-attention score set corresponding to the first matrix set; wherein the first self-attention score set and the elements in the first matrix set are in one-to-one correspondence;

[0074] performing a first weighting operation on each of the plurality of characteristic matrices in the first matrix set to obtain a weighted first matrix set;

[0075] The first self-attention score set and the weighted first matrix set are calculated to obtain a click probability value of one of the candidate products.

[0076] It should be noted that in the embodiment of the present application, based on the above content, the embodiment of the present application uses the self-attention mechanism to optimize multiple preset expert sub-models. Therefore, when calculating the click probability value of the candidate product, the following steps can be included:

[0077] First, a self-attention calculation is performed on a first feature matrix set according to a plurality of preset first query vectors to obtain a first self-attention score set corresponding to the first feature matrix set. Here, the preset first query vectors are in a one-to-one correspondence with the first feature matrix, and the preset first query vectors are learned from a large amount of sample data during model training. In addition, the first self-attention score set is in a one-to-one correspondence with the elements in the first matrix set.

[0078] Secondly, a first weighting operation is performed on each of the plurality of feature matrices in the first matrix set to obtain a weighted first matrix set; wherein the specific weights in the first weighting operation are also learned from a large amount of sample data during the model training process;

[0079] Finally, based on the first self-attention score set and the weighted first matrix set, the click probability value of one of the candidate products is calculated.

[0080] Similarly, in some embodiments, calculating the second matrix set using the target gating sub-model to obtain the purchase probability value of one of the candidate products may include:

[0081] Performing self-attention calculation on the second matrix set according to a plurality of preset second query vectors, obtaining a second self-attention score set corresponding to the second matrix set; wherein the second self-attention score set and the elements in the second matrix set are in one-to-one correspondence;

[0082] performing a first weighting operation on each of the plurality of characteristic matrices in the second matrix set to obtain a weighted second matrix set;

[0083] The second self-attention score set and the weighted second matrix set are calculated to obtain a purchase probability value of one of the candidate products.

[0084] It should be noted that when calculating the purchase probability value, the aforementioned click probability value can be referred to, which will not be elaborated here.

[0085] In general, by introducing the self-attention mechanism into the preset gated sub-model, the target expert sub-model is equivalent to projecting the input into different feature spaces, with each task favoring a different feature space. Leveraging the idea of ​​a query / key / value weight matrix set (Transformer), multiple query vectors are learned for each task, and the inner product of the query vector and the target expert sub-model is used as the self-attention score for the target expert sub-model. Because inner products can be positive or negative, the weighted feature vectors are merged and then reduced to the original space rather than directly added. In this way, by introducing the self-attention mechanism into the gated sub-model, the accuracy of the model can be further improved.

[0086] Furthermore, in some embodiments, each of the plurality of expert sub-models includes a plurality of deep learning hidden layers.

[0087] It should be noted that the more hidden layers deep learning has, the better it can learn high-order features. The embodiment of the present application improves the original MMoE multi-task model into a multi-layer Expert-Gate structure to further improve the accuracy of the model.

[0088] In summary, through the target recommendation model, the click probability value and purchase probability value of multiple candidate products are finally calculated.

[0089] S103: Calculating recommendation scores for each of the plurality of candidate products based on the click probability values ​​and purchase probability values ​​of each of the plurality of candidate products.

[0090] It should be noted that based on the click probability and purchase probability values ​​of multiple candidate products, a recommendation score for each candidate product can be calculated, and subsequent recommendation operations can be performed based on the recommendation score. Various algorithms can be used to calculate the recommendation score based on the click probability and purchase probability values, such as weighted summation, multiplication, and so on.

[0091] Furthermore, in some embodiments, calculating the recommendation score of each of the plurality of candidate products based on the click probability value and the purchase probability value of each of the plurality of candidate products includes:

[0092] Based on the preset weights, a weighted sum operation is performed on the click probability value and the purchase probability value of one of the candidate products to obtain a recommendation score for the one of the candidate products.

[0093] It should be noted that the click probability value and the purchase probability value are weighted and summed according to the preset weights, and finally the recommendation score of one of the candidate products is obtained.

[0094] S104: Recommending products to the user to be recommended based on the recommendation scores of the multiple candidate products.

[0095] It should be noted that product recommendations are made to the user based on the recommendation scores of the multiple candidate products. Generally speaking, candidate products with high recommendation scores are recommended to the user first. Here, some of the multiple candidate products can be selected and ranked by recommendation score before being recommended to the user, all products can be ranked by recommendation score before being recommended to the user, or only the products with the highest recommendation scores can be selected and recommended to the user.

[0096] The embodiment of the present application provides a product recommendation method, which obtains a plurality of feature data to be calculated corresponding to a plurality of candidate products; inputs the plurality of feature data to be calculated into a target recommendation model, and obtains the click probability value and purchase probability value of each of the plurality of candidate products output by the target recommendation model; wherein the target recommendation model is trained based on at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample; calculates the recommendation score of each of the plurality of candidate products based on the click probability value and purchase probability value of each of the plurality of candidate products; and recommends products to the user to be recommended based on the recommendation score of each of the plurality of candidate products. In this way, the target recommendation model can calculate a plurality of candidate products, thereby recommending products to the user to be recommended; in addition, since the target recommendation model is optimized in four aspects (Gate attention improvement, introduction of GDBT idea, Expert sharing improvement, and multi-level training), the accuracy of product recommendation is improved.

[0097] In another embodiment of the present application, see Figure 2 , which shows a flow chart of another commodity recommendation method provided by an embodiment of the present application. Figure 2 As shown, the method may include:

[0098] S201: Acquire multiple sample feature data, and determine target label values ​​for each of the multiple sample feature data in a preset label value set.

[0099] It should be noted that a method for training a target recommendation model is provided in the embodiments of this application. Here, the target recommendation model is trained based on sample feature data. Therefore, before model training, it is necessary to obtain multiple sample feature data and then determine the target label value of each of the multiple sample feature data from a preset label value set.

[0100] It should be noted that the target recommendation model is trained based on sample feature data. Therefore, before model training, it is necessary to obtain multiple sample feature data and then determine the target label value of each of the multiple sample feature data in the preset label value set.

[0101] Sample feature data is generally obtained from system log data. A specific sample feature data actually consists of the user features of a sample user and the product features of a product exposed to the sample user. Taking an e-commerce platform as an example, sample feature data can be extracted and processed from the system log. For example, based on the backend access record data of the e-commerce platform, the features of the user entering the website and the product features of the products that the user interacted with are obtained. After processing, sample feature data can be generated. It should be understood that after entering the website, the user may interact with multiple products, in which case multiple pieces of sample feature data will be generated.

[0102] Furthermore, in some embodiments, the sample feature data may include a user behavior feature subset, a user preference feature subset, a product granularity feature subset, and a product attribute feature subset.

[0103] In a specific embodiment, the user behavior feature subset may include browsed products, searched products, products added to the shopping cart, followed products, ordered products, etc.; the user preference feature subset may include brand preference, category preference, store preference, positive review rate preference, promotion preference, etc.; the product granularity feature subset may include single product granularity, three-level category granularity, brand granularity, store granularity, etc.; the product attribute feature subset may include product words, search words, brand words, modifiers, channel sources, prices, sales volume, extended attributes, etc.

[0104] Some of the main features are listed below: (1) The time since the user last clicked on a product; (2) The list of single products (Stock Keeping Unit, SKU) that the user last clicked on; (3) The relative difference between the current product price and the average price of products in the same third-level category that the user clicked on in the past; (4) The list of brands that the user last clicked on; (5) The list of third-level categories that the user last clicked on; (6) The list of stores that the user last clicked on; (7) The gender of the products that the user clicked on in the past hour; (8) The list of product words corresponding to the products that the user clicked on in the past day; (9) The list of products that the user has added to his or her attention in the past month; (10) Whether the user's gender is the same as the gender of the SKU.

[0105] It should be noted that the preset label values ​​(Label) are manually designed based on the application environment. In related technical solutions, there are generally only two label values, used to identify negative samples and positive samples respectively. In this embodiment, multiple preset label values ​​are set, which is more detailed and can improve the final training effect of the model.

[0106] Further, in some embodiments, the preset tag value set includes a first tag value, a second tag value, a third tag value, a fourth tag value and a fifth tag value; wherein, the first tag value indicates a no-click event, the second tag value indicates a low-quality click event, the third tag value indicates a high-quality click event, the fourth tag value indicates a low-quality order event, and the fifth tag value indicates a high-quality order event.

[0107] It should be noted that in the embodiment of the present application, the preset tag value set can be divided into five categories, namely the first tag value, the second tag value, the third tag value, the fourth tag value and the fifth tag value, and the first tag value, the second tag value, the third tag value, the fourth tag value and the fifth tag value correspond to no-click events, low-quality click events, high-quality click events, low-quality purchase events and high-quality purchase events. In addition, the embodiment of the present application only uses five categories of tag values ​​for illustration, but the number of categories of preset tag values ​​can be determined according to the specific usage environment, and the embodiment of the present application does not limit it.

[0108] Furthermore, in some embodiments, determining the target label value of each of the plurality of sample feature data in the preset label value set may include:

[0109] Obtaining a first sample user corresponding to first sample feature data; wherein the first sample feature data is any one of the plurality of sample feature data;

[0110] determining in sequence whether the first sample user has executed the no-click event, the low-quality click event, the high-quality click event, the low-quality order event, and the high-quality order event;

[0111] Determine a subset of label values ​​corresponding to the first sample feature data according to the judgment result;

[0112] The maximum label value in the label value subset is determined as the target label value of the first sample feature data.

[0113] It should be noted that, for a specific sample feature data (referred to as the first sample feature data), the steps for determining the target label value corresponding to the sample feature data are as follows:

[0114] First, obtain the first sample user corresponding to the first sample feature data;

[0115] Secondly, it is determined in sequence whether the first sample user has executed a no-click event, a low-quality click event, a high-quality click event, a low-quality order event, and a high-quality order event, thereby determining a label value subset corresponding to the first sample feature data in the preset label value set. It should be understood that during the first sample user's interaction with a product, multiple events (such as click events and purchase events) may be completed simultaneously. Therefore, the first sample feature data may correspond to multiple label values, and these multiple label values ​​constitute the label value subset.

[0116] Finally, the maximum label value in the label value subset is determined as the target label value of the first sample feature data. Generally speaking, the larger the label value, the more active the first sample user's behavior (such as purchase), and these sample users are the ones that the recommendation model should pay more attention to, so the maximum label value in the label value subset is generally taken as the target label value. It should be understood that the target label value determination rule is compatible with the definition rule of the preset label value. Therefore, the embodiment of the present application only provides a feasible target label value determination rule, and does not constitute a limitation on the target label value determination rule.

[0117] Here, it is generally necessary to obtain the first sample behavior data of the first sample user, so as to determine whether the first sample user has performed the aforementioned multiple events based on the first sample behavior data. Here, the first sample feature data essentially includes the user characteristics of the first sample user and the product characteristics of a certain product, and the first sample behavior data is the behavioral feature data when the first sample user interacts with the product, such as whether the first sample user clicked on the product, whether the first sample user opened the product details page, how long the first sample user browsed the product, etc.

[0118] Furthermore, in some embodiments, when determining whether the sample user has executed the low-quality click event and the high-quality click event, the method may further include:

[0119] Determining a user parent scene corresponding to the first sample user in the preset parent scene set;

[0120] Obtaining a preset sub-scenario set corresponding to the user parent scenario, and determining a user sub-scenario corresponding to a first sample user in the preset sub-scenario set;

[0121] Based on the user sub-scenario, obtaining conditions for occurrence of high-quality click events and conditions for occurrence of low-quality click events;

[0122] According to the occurrence conditions of the high-quality click event and the occurrence conditions of the low-quality click event, the determination of whether the sample user has executed the low-quality click event and the high-quality click event is performed.

[0123] It should be noted that for low-quality clicks and high-quality clicks, since the number of click events is relatively large and the causes are relatively complex (such as accidental clicks, interest, casual clicks, etc.), in order to better judge whether the clicks of the user are high-quality clicks or low-quality clicks, the scenario in which the first sample user is located can be determined based on the first sample behavior characteristics, and the specific definitions of high-quality clicks and low-quality clicks can be determined based on the scenario in which the first sample user is located, thereby improving the accuracy of the label.

[0124] Specifically, in an embodiment of the present application, the scenario is divided into multiple parent scenarios, and each parent scenario is further divided into multiple sub-scenarios. Based on the first sample behavior data, the user parent scenario corresponding to the first sample user can be determined, and then the user sub-scenario to which the first sample user ultimately corresponds can be determined. Here, the threshold conditions for judging high-quality clicks or low-quality clicks in each user sub-scenario are different. Therefore, based on the user sub-scenario corresponding to the first sample user, the high-quality click event definition and the low-quality click event definition are obtained, thereby completing the subsequent judgment process. In this way, the high / low-quality click event definition is adapted to the user sub-scenario to which the first sample user belongs, thereby improving the accuracy of the label value.

[0125] In a specific embodiment, the preset parent scenario set includes a repeat purchase scenario, a clear purpose scenario, and a strolling scenario; the preset sub-browsing scenario set includes an immediate purchase scenario, an interested scenario, and an accidental click scenario.

[0126] In this way, by introducing high-quality and low-quality label segmentation logic, sample feature data can be further subdivided into high-quality positive samples, low-quality positive samples, or negative samples. In the specific design of labels, high-quality and low-quality clicks are divided into sub-scenarios, which improves the accuracy of labels and thus enhances the effectiveness of subsequent model training.

[0127] S202: Determine at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample from the plurality of sample feature data based on respective target label values ​​of the plurality of sample feature data;

[0128] It should be noted that, based on the target label values ​​of the respective multiple sample feature data, at least one high-quality positive sample, at least one low-quality positive sample and at least one negative sample are determined from the multiple sample feature data.

[0129] In other words, the target label value of the sample feature data is used to indicate the training category of the sample feature data, which includes high-quality positive samples, low-quality positive samples, and negative samples. In other words, for positive samples, the embodiment of the application further subdivides them into high-quality positive samples and low-quality positive samples, so that subsequent model training will place greater emphasis on the characteristics of high-quality positive samples, thereby improving the accuracy of the model.

[0130] Furthermore, in some embodiments, before determining at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample from the plurality of sample feature data, the method further includes:

[0131] When the preset multi-task model is used to determine the click probability value, the sample feature data whose target label value is the first label value is determined as a negative sample; the sample feature data whose target label value is the second label value is determined as a low-quality positive sample; and the sample feature data whose target label value is the third label value, the fourth label value, or the fifth label value is determined as a high-quality positive sample;

[0132] When the preset multi-task model is used to determine the purchase probability value, the sample feature data whose target label value is the first label value, the second label value or the third label value is determined as a negative sample; the sample feature data whose target label value is the fourth label value is determined as a low-quality positive sample, and the sample feature data whose target label value is the fifth label value is determined as a high-quality positive sample.

[0133] It should be noted that the multi-task model can perform multiple different tasks at the same time. In the embodiment of the present application, the target recommendation model can perform click calculation tasks and purchase calculation tasks at the same time.

[0134] In other words, the preset multi-task model includes two calculation links, which are used to complete the click calculation task and the purchase calculation task respectively. The click calculation task is used to calculate the click probability value of the to-be-recommended user for a candidate product, and the purchase calculation task is used to calculate the purchase probability value of the to-be-recommended user for a candidate product. Afterwards, based on the determined click probability value and purchase probability value, the recommendation score of a candidate product relative to the to-be-recommended user is finally obtained.

[0135] At this point, the definitions of high-quality positive samples, low-quality positive samples, and negative samples vary for different tasks. Specifically, the five label values ​​correspond to no-click events, high-quality click events, low-quality click events, high-quality order events, and low-quality order events, respectively.

[0136] When the preset multi-task model is used to determine the click probability value, the first label value represents a negative sample, the second label value represents a low-quality positive sample, and the third label value, the fourth label value, and the fifth label value all represent high-quality positive samples.

[0137] When the preset multi-task model is used to determine the purchase probability value, the first label value, the second label value, and the third label value all represent negative samples, the fourth label value represents a low-quality positive sample, and the fifth label value all represents a high-quality positive sample.

[0138] In this way, at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample are determined from the plurality of sample feature data, so that subsequent model training can be performed based on these samples.

[0139] S203: Training a preset multi-task model using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample to obtain a target recommendation model.

[0140] It should be noted that after determining the target label value of each sample feature data, the preset multi-task model can be trained using the at least one high-quality positive sample, at least one low-quality positive sample and at least one negative sample to obtain a target recommendation model.

[0141] Here, the preset multi-task model may include models based on various principles, such as the Multi-Gate Mixture-of-Experts (MMoE) multi-task model, the Deep Factorization Machines (DeepFM) multi-task learning model, the Mixture of Sequential Experts (MoSE) multi-task model, the Mixture-of-Experts (MoE) multi-task model, etc., which are not limited in the embodiments of the present application. However, for the sake of convenience, the following content is explained using the MMoE multi-task model as an example.

[0142] Furthermore, due to the introduction of label segmentation logic, high-quality positive samples and low-quality positive samples are further segmented in the positive samples, so corresponding weights can be determined for high-quality positive samples and low-quality positive samples respectively, thereby weighting different samples. In this case, the method of training the preset multi-task model using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample to obtain the target recommendation model may include:

[0143] Determining a weight coefficient for each of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample according to each of the target label values ​​of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample;

[0144] Constructing a loss function of the preset multi-task model based on the weight coefficients of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample;

[0145] The preset multi-task model is trained using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample. When the result of the loss function reaches a preset threshold, the trained preset multi-task model is determined as the target recommendation model.

[0146] It should be noted that, based on the target label values ​​of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample, weight coefficients for the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample can be determined, respectively. Here, the corresponding relationship between the weight coefficients and the target label values ​​is preset.

[0147] Next, based on the respective weight coefficients of at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample, the loss function of the preset multi-task model is constructed. At this time, each sample receives different attention when calculating the loss function due to its different weight coefficient. That is, in the loss function of model training, different label weights are set for sample weighting. Specifically, according to the target label value, samples with positive behaviors (such as high-quality purchasing behaviors) can be determined. Compared with samples with other behaviors, these samples with positive behaviors should be paid more attention, so these samples with positive behaviors can be given greater weights. Taking the aforementioned label value scheme as an example, the sample with a label value of 4 can be given the largest weight value, and the sample with a label value of 0 can be given the smallest weight value.

[0148] Finally, using the at least one high-quality positive sample, at least one low-quality positive sample and at least one negative sample, when the result of the loss function is lower than a preset threshold (or converges to a certain value), the trained preset multi-task model is determined as the target recommendation model.

[0149] It should also be noted that the MMoE multi-task model includes multiple expert sub-models (abbreviated as Expert) and gating sub-models (abbreviated as Gate). The multiple expert sub-models respectively output multiple feature matrices. The gating sub-model performs weighted calculations on the multiple feature matrices according to different tasks, thereby obtaining click probability values ​​and purchase probability values ​​respectively. For the MMoE multi-task model, the gating sub-model actually includes multiple. For the sake of convenience, the embodiments of this application are collectively referred to as "gating sub-models".

[0150] Therefore, in some embodiments, the preset multi-task model includes at least one first preset expert sub-model, at least one second preset expert sub-model and at least one shared preset expert sub-model; wherein,

[0151] The at least one first preset expert sub-model and the at least one shared preset expert sub-model are used to determine a click probability value;

[0152] The at least one second preset expert sub-model and the at least one shared preset expert sub-model are used to determine a purchase probability value.

[0153] It should be noted that, based on the foregoing, the preset multi-task model can include click calculation tasks and purchase calculation tasks. In related art, multiple expert sub-models in the MMoE multi-task model are shared by different tasks, with the gating sub-models establishing different weights for different tasks. In the embodiment of the present application, the multiple preset expert sub-models are divided into a first preset expert sub-model, a second preset expert sub-model, and a shared preset expert model.

[0154] Here, the first preset expert sub-model and the shared preset expert model are combined to calculate the click probability value, and the second preset expert sub-model and the shared preset expert model are combined to calculate the purchase probability value. In other words, this application subdivides the preset expert sub-model into task-specific experts and shared experts to further better characterize the relevance and differences between different tasks.

[0155] In order to further improve the accuracy of the target recommendation model, the embodiments of the present application have made some optimizations to the MMoE multi-task model. Therefore, in some embodiments, before training the preset multi-task model using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample, the method further includes:

[0156] The at least one first preset expert sub-model, the at least one second preset expert sub-model and the at least one shared preset expert sub-model are optimized using a gradient boosting tree model.

[0157] It should be noted that in the relevant technical solutions, the relationship between multiple expert sub-models in the MMoE multi-task model can be viewed as a random forest. Each expert sub-model gives its own suggestions and contributes to the entire model in an integrated way. However, for tree models, the gradient boosting tree model (Gradient Boosting Decision Tree, GBDT) serial approach is better than the random forest parallel approach. Therefore, the embodiment of the present application uses the gradient boosting tree serial mechanism to optimize multiple preset expert sub-models.

[0158] Specifically, for multiple expert sub-models, the latter expert sub-model takes the sample feature data and the output of the previous expert sub-model as input, thereby improving the accuracy of the final model.

[0159] Furthermore, in some embodiments, the preset multi-task model includes a preset gating sub-model; before training the preset multi-task model using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample, the method may further include:

[0160] The preset gating sub-model is optimized using a self-attention model.

[0161] It should be noted that the preset multi-task model also includes a preset gating sub-model, so as to weight the output of the preset expert sub-model according to different tasks, and finally calculate the click probability value or the purchase probability value. In the related art, the MMoE multi-task model only learns a feature weight for each task, and then linearly weights the output of the expert sub-model, that is, the gating sub-model part only undergoes one linear transformation, and fails to fully learn the sample feature distribution. At the same time, the gating sub-model is strongly correlated with the sub-task and should be closer to the output rather than the input. Therefore, in an embodiment of the present application, the preset gating sub-model is optimized using a self-attention model (attention, also called a self-attention mechanism), and the expert sub-model is equivalent to projecting the input into different feature spaces, and each task is biased towards a different feature space, which further improves the accuracy of the model.

[0162] Furthermore, in some embodiments, each of the plurality of expert sub-models includes a plurality of deep learning hidden layers.

[0163] It should be noted that the more hidden layers a deep learning algorithm has, the better it can learn high-order features. The embodiment of the present application improves the original MMoE multi-task model into a multi-layer Expert-Gate structure to further improve the accuracy of the model.

[0164] The embodiment of the present application provides a product recommendation method. Through the detailed explanation of the aforementioned embodiment in this embodiment, it can be seen that the preset multi-task model is trained according to the sample feature data, so that the preset multi-task model learns more in-depth features and finally obtains a more accurate target recommendation model; at the same time, due to the introduction of the subdivision logic of high-quality positive samples, low-quality positive samples and negative samples in terms of label values, and the sub-scene division of high-quality and low-quality clicks, the accuracy of the target recommendation model is further improved; in addition, for the preset multi-task model, four optimization points are introduced (Gate attention improvement, introduction of GBDT idea, Expert sharing improvement and multi-level training), thereby further improving the accuracy of the target recommendation model.

[0165] In another embodiment of the present application, taking a certain express version application as an example, the aforementioned model training and product recommendation method is applied to the homepage sorting scenario, see Figure 3 , which shows a schematic diagram of an application scenario of a product recommendation method provided in an embodiment of the present application.

[0166] like Figure 3As shown, in this application scenario, the following sections are included: Express version homepage, Express version merchant details (product details), Express version payment completion (Simple payment completion), order details, order tracking, shopping cart and Express version of My JD (referred to as Express version of My JD). Specifically, the closed-loop link of a certain express version application is as follows: (1) The express version homepage is aimed at conversion, allowing users to discover products of interest; (2) The express version product details are divided into the middle and bottom parts. The middle part is aimed at conversion, allowing users to quickly view similar and related products of the main product (Stock Keeping Unit, SKU). The bottom part is based on the middle part and allows users to discover other products of interest; (3) The payment completion page is aimed at conversion, allowing users to quickly discover products that can be combined with the main SKU; (4) The order details page is aimed at conversion, allowing users to quickly discover products that can be combined with the main SKU; (5) The order tracking page is aimed at conversion, allowing users to quickly discover products that can be combined with the main SKU; (6) The express version of My JD is aimed at conversion, allowing users to quickly discover products of interest; (7) The shopping cart is aimed at conversion. If there are add-on products, it mainly allows users to discover related products that can be purchased with the main SKU. If there are no add-on products, it mainly allows users to discover products of interest.

[0167] Based on the business closed loop of a certain express version application, see Figure 4 , which shows a structural diagram of a product recommendation system provided by an embodiment of the present application. Figure 4 As shown, the product recommendation system provided in the embodiment of the present application can be composed of the following six modules: business positioning module 301, feature mining module 302, label design module 303, multi-task learning model improvement module 304, offline evaluation index module 305 and online iteration module 306.

[0168] The express version of the business positioning module 301 includes the following:

[0169] (1) Downstream scenarios: The Express Edition is a lightweight application (Application, APP) for downstream users. To emphasize the downstream characteristics of the Express Edition, the recommendation strategy of the Express Edition needs to be based on the independent downstream scenarios of the Express Edition, so as to better identify the needs of the Express Edition users and accurately match people with goods. Low price is one of the core competitiveness and core differentiation points of the Express Edition. Therefore, the method of reducing postage and combining low-priced goods is adopted to achieve the positioning of truly low-priced good goods. Based on the recommendation model, the free shipping factor and the user free shipping sensitivity factor are introduced to increase the probability of recommending free shipping products to users who are sensitive to free shipping. In order to better emphasize the low price characteristics of the Express Edition and the degree of matching between the overall products and users, the Express Edition launched an exclusive channel price, creating an overall downstream e-commerce atmosphere with low-priced and high-quality products. In addition, the Express Edition also launched a gold coin activity for browsing products, allowing users to do some small tasks while browsing products, hoping to generate immediate purchase behavior during the task. "Making money" is one of the main positionings of the Speed ​​Version. There are a large number of game and task resource slots in the Speed ​​Version APP, including games and tasks that allow you to earn coins by browsing products, participating in activities, and watching videos. In the initial stage, the products in the game resource slots are displayed randomly, and later on, they will be connected to the recommendation function for precise distribution.

[0170] (2) Expected benefits: Adjust the recommendation strategy to align with Yewufang's long-term strategy of sinking into the lower-tier markets. The initial goal is to increase the click-through rate of exposed users. As the business iterates, the exposure user conversion index and user (unique visitor, UV) value index must also be taken into account.

[0171] (3) Optimization of homepage recommendations: The express version homepage recommends to you an increase in the weight of low-priced goods based on recall, sorting, and diversity. This embodiment of the application has made optimizations in both the recall stage and the sorting stage;

[0172] (4) User experience optimization: (a) The Express Edition adds exposure negative feedback filtering logic for all scenarios. The Express Edition sometimes recommends products that users are not interested in, and the initial Express Edition has no negative feedback channel, which has a great impact on the user experience. Therefore, new exposure filtering for each scenario has been added. It mainly includes: out-of-stock filtering, purchased filtering, sensitive word filtering, price filtering, season filtering, same-image filtering, purchasing power filtering, etc.; (b) Gender filtering of product titles in the Express Edition. There are a large number of gender-related cases in the Express Edition product experience. Female products such as stockings and thongs are introduced to male users who have no related behavior, which has a significant impact on the user experience. In order to improve the user experience, during the recall stage, for some recall sources that are weakly related to behavior, according to the SKU title, if there are strong gender characters in the title, such as "male" and "female", these products will be filtered.

[0173] The feature mining module 302 is mainly used to pre-process user data to obtain sample feature data, including the following:

[0174] (1) User behavior profile (used to obtain a subset of user behavior features): User behavior mainly includes browsed products, searched products, added products to cart, followed products, ordered products, etc.

[0175] (2) User preference profile (used to obtain a subset of user preference features): User preferences mainly include brand preference, category preference, store preference, positive review preference, promotion preference, etc.

[0176] (3) Product granularity portrait (used to obtain a subset of product granularity features): Product granularity mainly includes SKU granularity, three-level category granularity, brand granularity, store granularity, etc.

[0177] (4) Product attribute portrait (used to obtain a subset of product attribute features): Product attributes mainly include product words, search words, brand words, modifiers, channel sources, prices, sales volume, extended attributes, etc.

[0178] Some of the main features are listed below: (1) The time since the user last clicked on a product; (2) The list of SKUs that the user last clicked on; (3) The relative difference between the current product price and the average price of products in the same third-level category that the user clicked on in the past; (4) The list of brands that the user last clicked on; (5) The list of third-level categories that the user last clicked on; (6) The list of stores that the user last clicked on; (7) The gender of the products that the user clicked on in the past hour; (8) The list of product words corresponding to the products that the user clicked on in the past day; (9) The list of products that the user has added to his or her attention in the past month; (10) Whether the user's gender and SKU gender are the same.

[0179] The label design module 303 is mainly used to determine the target label value corresponding to the sample feature data, including the following contents:

[0180] (1) Bottom-layer Label table logic: Based on the user's behavior in the JD system, the server reports the behavior in the feature log. If the behavior is exposed to the user but the user does not click on it, the label is 0; if the user clicks on it with low quality, the label is 1; if the user clicks on it with high quality, the label is 2; if the user purchases it with low quality, the label is 3; and if the user purchases it with high quality, the label is 4.

[0181] (2) Definition of high-quality and low-quality click events: Since the amount of data for click events is large, in order to more accurately segment whether the click is a high-quality click event or a low-quality click event. Therefore, for the important scenario where the user enters the product details page, a series of session behaviors of the user from entering to leaving the product details page are recorded to determine whether it is a valuable click. Users may reach the product details page through various channels with different intentions. The definition of high-quality click events in the embodiments of the present application varies according to the different scenarios in which the user is located. The specific scenarios are divided into three parent scenarios: repeat purchase, clear purpose, and strolling. Each parent scenario is further divided into three sub-scenarios: immediate purchase, interest, and accidental click.

[0182] Here, the definitions of the three sub-scenarios of immediate purchase, interest, and accidental click are as follows:

[0183] A. Parent scenario 1 - Repeat purchase: The user has purchased the product within one year, or the click comes from the order page.

[0184] (a) Sub-scenario 1 - Buy Now: Click events include purchase intent events such as Buy Now and Add to Cart.

[0185] (b) Sub-scenario 2 - Interested: Does not include purchase intent events, and the dwell time is greater than or equal to 30 seconds or the number of included events is greater than or equal to 3;

[0186] (c) Sub-scenario 3 - Accidental clicks: does not include purchase intent events, and the dwell time is less than 30 seconds and the number of included events is less than 3.

[0187] B. Parent scenario 2 - clear purpose: clicks come from search results page, shopping cart page, follow page, etc.

[0188] C. Parent scene 3 - Browsing: Clicks come from other sources.

[0189] This embodiment of the application sets reasonable thresholds and, based on the business context, debugs the data distribution of high- and low-quality click events, ultimately obtaining high- and low-quality click event definitions corresponding to each sub-scenario. In actual use, based on the user's behavioral data, the parent scenario the user is in is determined, and then the sub-scenario the user is in is determined. Based on the sub-scenario the corresponding high- and low-quality click event definitions are then obtained, and the user's click event is determined to be a high-quality or low-quality click event.

[0190] (3) Definition of high-quality and low-quality order events: A high-quality order event is defined as a high-quality order when a user places an order and completes the entire purchase process; a low-quality order event is defined as a user placing an order but then canceling it.

[0191] (4) Label refinement: The role of label refinement is to set different label weights for sample weighting in the loss function of model training.

[0192] For the multi-task learning model improvement module 304, the multi-task learning MMoE multi-task model used in the prior art includes an expert sub-model (Expert, or called an expert sub-network) and a gated sub-model (Gate, or called a gated sub-network). Each sub-network is implemented by a deep neural network (DNN). The expert sub-model is weighted by the gated sub-model, and there is no direct interaction between multiple expert sub-models. The embodiment of the present application has made improvements to it from the perspective of expert interaction and deep expert sharing, mainly including the following contents:

[0193] (1) Gate attention improvement: The original MMoE multi-task model learns a feature weight for each task based on the Mixture-of-Experts (MOE) model, and linearly weights the features extracted by the expert sub-model. The gated sub-model only undergoes one linear transformation and fails to fully learn the sample distribution. At the same time, the gated sub-model is strongly correlated with the sub-task and should be closer to the output rather than the input. On this basis, the embodiment of the present application introduces a self-attention mechanism in the gated sub-model part. The expert sub-model is equivalent to projecting the input into different feature spaces, and each task is biased towards a different feature space. With the help of the transformer idea, multiple query vectors are learned for each task (each query vector corresponds to an expert sub-model), and the inner product of the query vector and the corresponding expert sub-model output is used as the self-attention score of the expert sub-model. Since the inner product can be positive or negative, the weighted feature vectors are merged and then reduced to the original space instead of directly adding them. After the model was launched, the average click rate of exposed users increased by 1.2%.

[0194] (2) Introducing the GBDT concept: The relationship between multiple experts in the original MMoE multi-task model can be viewed as a random forest. Each expert provides his or her own suggestions, which are contributed to the entire network in an integrated manner. Since the serial method of the gradient boosting tree model is more effective than the parallel method of the random forest in the tree model, the embodiment of this application improves the expert sub-model to a serial mode. The latter expert sub-model takes the sample feature data and the output of the previous expert sub-model as the input. After the model was launched, the average click-through rate of exposed users increased by 0.88%, and the browsing depth increased by 1.23%.

[0195] (3) Expert Sharing Improvement: In the original MMoE multi-task model, multiple experts were shared across different tasks, with different gate weights established for each task. This embodiment of the application further subdivides experts into task-specific experts and shared experts, further characterizing the relevance and distinctions between different tasks. After the model was launched, the click-through rate of exposed users increased by 2%.

[0196] (4) Multi-level training: The more hidden layers a deep learning model has, the better it can learn high-level features. The embodiment of the present application improves the original MMoE multi-task model into a multi-layer Expert-Gate structure.

[0197] The offline evaluation index module 305 mainly includes the following contents:

[0198] (1) Exposure user click value: Exposure user click value = number of clicks / exposure UV, where exposure UV is the number of exposed users after deduplication by browser;

[0199] (2) Exposure user conversion value: Exposure user conversion value = order line / exposure UV;

[0200] (3) Browsing depth: Browsing depth = number of products browsed by the user / number of products exposed to the user;

[0201] (4) User UV value: User UV value = GMV / Exposure UV, where GMV refers to the total amount contributed by users.

[0202] The online iteration module 306 mainly includes the following contents:

[0203] (1) Multi-party joint debugging: Before the product recommendation model of the embodiment of the present application was launched, data engineers, algorithm engineers, R&D engineers, product engineers, and operations engineers conducted multi-party joint debugging to ensure that the logic was correct and the performance was acceptable before the flash;

[0204] (2) Testing the whitelist: In order to detect problems in advance and in a timely manner, the product recommendation model of the embodiment of the present application is implemented online, and a whitelist is configured to test the effect of recommended products and user experience;

[0205] (3) Online observation: After the product recommendation model of the embodiment of the present application is launched, the real-time effect, online service performance, recommendation position availability and other indicators are observed to ensure that everything is normal;

[0206] (4) Optimization and iteration: With the continuous updating of business, the evaluation indicators of home page recommendations are constantly upgraded. The embodiment of this application optimizes the iterative model according to the new needs and improves the online effect.

[0207] In short, the embodiment of this application introduces the logic of high- and low-quality label segmentation, which brings significant improvement compared to the previous label definition. The four improvements of the multi-task learning model: Gate attention improvement, introduction of GDBT ideas, Expert sharing improvement, and multi-level training have been successfully applied online and brought about indicator improvement. In addition, the embodiment of this application explores a variety of offline evaluation indicators, which are of guiding significance for the alignment of online and offline model experiments.

[0208] The embodiment of the present application provides a product recommendation method. Through the detailed explanation of the aforementioned embodiment in this embodiment, it can be seen that the preset multi-task model is trained according to the sample feature data, so that the preset multi-task model learns more in-depth features and finally obtains a more accurate target recommendation model; at the same time, due to the introduction of the subdivision logic of high-quality positive samples, low-quality positive samples and negative samples in terms of label values, and the sub-scene division of high-quality and low-quality clicks, the accuracy of the target recommendation model is further improved; in addition, for the preset multi-task model, four optimization points are introduced (Gate attention improvement, introduction of GBDT idea, Expert sharing improvement, multi-level training), thereby further improving the accuracy of the target recommendation model.

[0209] In yet another embodiment of the present application, see Figure 5 , which shows a structural diagram of a product recommendation device 40 provided in an embodiment of the present application, such as Figure 5 As shown, the product recommendation device 40 includes a data unit 401, a calculation unit 402, a determination unit 403 and a recommendation unit 404; wherein,

[0210] The data unit 401 is configured to obtain a plurality of feature data to be calculated corresponding to a plurality of candidate products;

[0211] A calculation unit 402 is configured to input the plurality of feature data to be calculated into a target recommendation model, and obtain a click probability value and a purchase probability value for each of the plurality of candidate products output by the target recommendation model; wherein the target recommendation model is trained based on at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample;

[0212] The determination unit 403 is configured to calculate a recommendation score for each of the plurality of candidate products based on the click probability value and the purchase probability value of each of the plurality of candidate products;

[0213] The recommendation unit 404 is configured to recommend products to the user to be recommended based on the recommendation scores of the plurality of candidate products.

[0214] In some embodiments, the target recommendation model includes at least one first target expert model, at least one second target expert model, at least one shared target expert model and a target gating sub-model; the calculation unit 402 is specifically configured to use at least one first target expert model, at least one second target expert model and at least one shared target expert model to calculate the multiple feature data to be calculated respectively, and obtain at least one first output result, at least one second output result and at least one shared output result; based on the first output result and the shared output result, the click probability value of one of the candidate products is obtained through the target gating sub-model; based on the second output result and the shared output result, the purchase probability value of one of the candidate products is obtained through the target gating sub-model.

[0215] In some embodiments, the determination unit 403 is specifically configured to perform a weighted sum operation on the click probability value and the purchase probability value of one of the candidate products based on preset weights to obtain a recommendation score for the one of the candidate products.

[0216] In some embodiments, as Figure 6 As shown, the product recommendation device 40 also includes a training unit 405, which is configured to obtain multiple sample feature data and determine the target label value of each of the multiple sample feature data in a preset label value set; based on the target label value of each of the multiple sample feature data, determine at least one high-quality positive sample, at least one low-quality positive sample and at least one negative sample from the multiple sample feature data; use the at least one high-quality positive sample, at least one low-quality positive sample and at least one negative sample to train a preset multi-task model to obtain a target recommendation model.

[0217] In some embodiments, the training unit 405 is further configured to determine the weight coefficients of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample based on their respective target label values; construct a loss function of the preset multi-task model based on the weight coefficients of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample; train the preset multi-task model using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample, and when the result of the loss function reaches a preset threshold, determine the trained preset multi-task model as the target recommendation model.

[0218] In some embodiments, the preset tag value set includes a first tag value, a second tag value, a third tag value, a fourth tag value and a fifth tag value; wherein, the first tag value indicates a no-click event, the second tag value indicates a low-quality click event, the third tag value indicates a high-quality click event, the fourth tag value indicates a low-quality order event, and the fifth tag value indicates a high-quality order event.

[0219] In some embodiments, the training unit 405 is further configured to obtain a first sample user corresponding to first sample feature data; wherein the first sample feature data is any one of the multiple sample feature data; determine in turn whether the first sample user has executed the no-click event, the low-quality click event, the high-quality click event, the low-quality order event, and the high-quality order event; determine a subset of label values ​​corresponding to the first sample feature data based on the judgment result; and determine the maximum label value in the label value subset as the target label value of the first sample feature data.

[0220] In some embodiments, the training unit 405 is further configured to, when the preset multi-task model is used to determine the click probability value, determine the sample feature data of the target label value being the first label value as a negative sample; determine the sample feature data of the target label value being the second label value as a low-quality positive sample, and determine the sample feature data of the target label value being the third label value, the fourth label value or the fifth label value as a high-quality positive sample; when the preset multi-task model is used to determine the purchase probability value, determine the sample feature data of the target label value being the first label value, the second label value or the third label value as a negative sample; determine the sample feature data of the target label value being the fourth label value as a low-quality positive sample, and determine the sample feature data of the target label value being the fifth label value as a high-quality positive sample.

[0221] In some embodiments, the preset multi-task model includes at least one first preset expert sub-model, at least one second preset expert sub-model and at least one shared preset expert sub-model; wherein, the at least one first preset expert sub-model and at least one shared preset expert sub-model are used to determine the click probability value; the at least one second preset expert sub-model and at least one shared preset expert sub-model are used to determine the purchase probability value.

[0222] In some embodiments, the preset multi-task model includes multiple preset expert sub-models; the training unit 405 is also configured to use a gradient boosting tree model to perform optimization operations on the at least one first preset expert sub-model, at least one second preset expert sub-model and at least one shared preset expert sub-model.

[0223] In some embodiments, the preset multi-task model includes a preset gating sub-model; the training unit 405 is further configured to optimize the preset gating sub-model using a self-attention model.

[0224] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0225] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer storage medium, which is applied to the product recommendation device 40 and stores a computer program. When the computer program is executed by a processor, it implements any of the methods described in the aforementioned embodiments.

[0226] Based on the composition of the above-mentioned commodity recommendation device 40 and the computer storage medium, see Figure 7 , which shows a specific hardware structure example of the product recommendation device 40 provided in an embodiment of the present application, such as Figure 7 As shown, the product recommendation device 40 may include: a communication interface 501, a memory 502 and a processor 503; each component is coupled together via a bus system 504. It is understood that the bus system 504 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 504 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 7 In FIG, various buses are labeled as bus system 504.

[0227] Communication interface 501, used for sending and receiving signals during the process of sending and receiving information with other external network elements;

[0228] Memory 502, used to store computer programs that can be run on processor 503;

[0229] The processor 503 is configured to, when running the computer program, execute:

[0230] Obtain multiple feature data to be calculated corresponding to multiple candidate products;

[0231] Inputting the plurality of feature data to be calculated into a target recommendation model, and obtaining click probability values ​​and purchase probability values ​​of the plurality of candidate products output by the target recommendation model; wherein the target recommendation model is trained based on at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample;

[0232] Calculating a recommendation score for each of the plurality of candidate products based on the click probability value and the purchase probability value of each of the plurality of candidate products;

[0233] Recommend products to the user to be recommended based on the recommendation scores of the multiple candidate products.

[0234] It is understood that the memory 502 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 502 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0235] The processor 503 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 503 or by software instructions. The processor 503 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 502, and the processor 503 reads the information in the memory 502 and, in conjunction with its hardware, completes the steps of the above method.

[0236] It is understood that the embodiments described herein may be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, or other electronic units or combinations thereof for performing the functions described herein.

[0237] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0238] Optionally, as another embodiment, the processor 503 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0239] Based on the composition and hardware structure example of the above-mentioned commodity recommendation device 40, see Figure 8 , which shows a structural diagram of another product recommendation system 60 provided in an embodiment of the present application.

[0240] like Figure 8 As shown, the product recommendation system 60 includes at least the product recommendation device 40 described in any one of the aforementioned embodiments, which can train the preset multi-task model based on the sample feature data, so that the preset multi-task model can learn more in-depth features and finally obtain a more accurate target recommendation model; at the same time, the segmentation logic of high-quality positive samples, low-quality positive samples and negative samples is introduced in terms of label values, which further improves the accuracy of the target recommendation model; in addition, for the preset multi-task model, four optimization points are introduced (Gate attention improvement, introduction of GBDT ideas, Expert sharing improvement and multi-level training), thereby further improving the accuracy of the target recommendation model. At this time, the product recommendation device 40 uses the target recommendation model to recommend products to the recommended user, which can also improve the accuracy of the product recommendation.

[0241] The above description is merely a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application.

[0242] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0243] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0244] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0245] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0246] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0247] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A product recommendation method, characterized in that: The method comprises: Obtain multiple feature data to be calculated corresponding to multiple candidate products; Input the multiple feature data to be calculated into a target recommendation model, and obtain click probability values ​​and purchase probability values ​​of each of the multiple candidate products output by the target recommendation model; wherein the target recommendation model is obtained by training a preset multi-task model based on at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample; wherein the preset multi-task model is used to complete the tasks of click probability values ​​and purchase probability values, and for different tasks, the definitions of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample are different; when the preset multi-task model is used to determine the click probability value, the sample feature data with a target label value of the first label value is determined as a negative sample; the sample feature data with a target label value of the second label value is determined as a low-quality positive sample, and the sample feature data with a target label value of the third label value, the fourth label value, or the fifth label value is determined as a high-quality positive sample; when the preset multi-task model is used to determine the purchase probability value, the sample feature data with a target label value of the first label value, the second label value, or the third label value is determined as a negative sample; the sample feature data with a target label value of the fourth label value is determined as a low-quality positive sample, and the sample feature data with a target label value of the fifth label value is determined as a high-quality positive sample; Calculating a recommendation score for each of the plurality of candidate products based on the click probability value and the purchase probability value of each of the plurality of candidate products; Recommend products to the user to be recommended based on the recommendation scores of the multiple candidate products.

2. The product recommendation method according to claim 1, characterized in that: The target recommendation model includes at least one first target expert model, at least one second target expert model and at least one shared target expert model and a target gating sub-model; Inputting the plurality of feature data to be calculated into a target recommendation model, and obtaining click probability values ​​and purchase probability values ​​of the plurality of candidate products output by the target recommendation model, includes: Calculating one of the feature data to be calculated using at least one first target expert model, at least one second target expert model, and at least one shared target expert model, respectively, to obtain at least one first output result, at least one second output result, and at least one shared output result; Based on the at least one first output result and the at least one shared output result, calculating a click probability value of one of the candidate products through the target gating sub-model; Based on the at least one second output result and the at least one shared output result, a purchase probability value of one of the candidate commodities is calculated by the target gating sub-model.

3. The product recommendation method according to claim 1, characterized in that: Calculating the recommendation scores of the plurality of candidate products according to the click probability values ​​and purchase probability values ​​of the plurality of candidate products includes: Based on the preset weights, a weighted sum operation is performed on the click probability value and the purchase probability value of one of the candidate products to obtain a recommendation score for the one of the candidate products.

4. The product recommendation method according to claim 1, wherein: The method further comprises: Acquire multiple sample feature data, and determine target label values ​​for each of the multiple sample feature data in a preset label value set; Determining at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample from the plurality of sample feature data based on respective target label values ​​of the plurality of sample feature data; The preset multi-task model is trained using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample to obtain a target recommendation model.

5. The product recommendation method according to claim 4, characterized in that: The method of training a preset multi-task model using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample to obtain a target recommendation model includes: Determining a weight coefficient for each of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample according to each of the target label values ​​of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample; Constructing a loss function of the preset multi-task model based on the weight coefficients of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample; The preset multi-task model is trained using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample. When the result of the loss function reaches a preset threshold, the trained preset multi-task model is determined as the target recommendation model.

6. The product recommendation method according to claim 4, characterized in that: The preset tag value set includes the first tag value, the second tag value, the third tag value, the fourth tag value and the fifth tag value; wherein, the first tag value indicates a no-click event, the second tag value indicates a low-quality click event, the third tag value indicates a high-quality click event, the fourth tag value indicates a low-quality order event, and the fifth tag value indicates a high-quality order event.

7. The product recommendation method according to claim 6, characterized in that: Determining the target label value of each of the plurality of sample feature data in the preset label value set includes: Obtaining a first sample user corresponding to first sample feature data; wherein the first sample feature data is any one of the plurality of sample feature data; determining in sequence whether the first sample user has executed the no-click event, the low-quality click event, the high-quality click event, the low-quality order event, and the high-quality order event; Determine a subset of label values ​​corresponding to the first sample feature data according to the judgment result; The maximum label value in the label value subset is determined as the target label value of the first sample feature data.

8. The product recommendation method according to claim 1, characterized in that: The preset multi-task model includes at least one first preset expert sub-model, at least one second preset expert sub-model and at least one shared preset expert sub-model; wherein, The at least one first preset expert sub-model and the at least one shared preset expert sub-model are used to determine the click probability value; The at least one second preset expert sub-model and the at least one shared preset expert sub-model are used to determine the purchase probability value.

9. The product recommendation method according to claim 8, characterized in that: Before training the preset multi-task model using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample, the method further includes: The at least one first preset expert sub-model, the at least one second preset expert sub-model and the at least one shared preset expert sub-model are optimized using a gradient boosting tree model.

10. The product recommendation method according to claim 4, characterized in that: The preset multi-task model includes a preset gating sub-model; before training the preset multi-task model using the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample, the method further includes: The preset gating sub-model is optimized using a self-attention model.

11. A product recommendation device, characterized in that: The product recommendation device includes a data unit, a calculation unit, a determination unit, a recommendation unit and a training unit; wherein, The data unit is configured to obtain a plurality of feature data to be calculated corresponding to a plurality of candidate products; The calculation unit is configured to input the plurality of feature data to be calculated into a target recommendation model, and obtain a click probability value and a purchase probability value for each of the plurality of candidate products output by the target recommendation model; wherein the target recommendation model is obtained by training a preset multi-task model based on at least one high-quality positive sample, at least one low-quality positive sample, and at least one negative sample; wherein the preset multi-task model is used to complete the task of calculating the click probability value and the purchase probability value, and the definitions of the at least one high-quality positive sample, the at least one low-quality positive sample, and the at least one negative sample are different for different tasks; The determining unit is configured to calculate a recommendation score for each of the plurality of candidate products based on the click probability value and the purchase probability value of each of the plurality of candidate products; The recommendation unit is configured to recommend products to the user to be recommended based on the recommendation scores of the multiple candidate products; The training unit is configured to, when the preset multi-task model is used to determine the click probability value, determine the sample feature data of the target label value being the first label value as a negative sample; determine the sample feature data of the target label value being the second label value as a low-quality positive sample, and determine the sample feature data of the target label value being the third label value, the fourth label value or the fifth label value as a high-quality positive sample; when the preset multi-task model is used to determine the purchase probability value, determine the sample feature data of the target label value being the first label value, the second label value or the third label value as a negative sample; determine the sample feature data of the target label value being the fourth label value as a low-quality positive sample, and determine the sample feature data of the target label value being the fifth label value as a high-quality positive sample.

12. A product recommendation device, characterized in that The product recommendation device includes a memory and a processor; wherein, The memory is used to store a computer program that can be run on the processor; The processor is configured to execute the steps of the method according to any one of claims 1 to 10 when running the computer program.

13. A computer storage medium, characterized in that The computer storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A product recommendation system, characterized in that: The product recommendation system at least includes the product recommendation device according to claim 11 or 12.

Images