Advertisement sorting method and device, electronic equipment and storage medium

By constructing directed graphs and random walk technology with weights, combined with advertising cold-start edge information, the problem of machine learning model processing single features on the ad side is solved, and more accurate advertising sorting and recommendation effects are achieved.

CN119090559APending Publication Date: 2024-12-06BEIJING QIYI CENTURY SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411212370.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing machine learning model is too single when processing advertising-side features and cannot effectively reflect user behavior preferences, resulting in low ad click-through rates and conversion rates.

Method used

By constructing a directed graph with weights, using the user sequence behavior in the time window to randomly walk, multiple ad sequences are obtained, and inputting the edge information of the advertisement cold start to the graph into the model, obtaining the graph embedding vector, and processing is The feature vector of the fine-sorting model, used for advertising sorting.

Benefits of technology

This method introduces richer information, makes full use of user behavior and advertising ID characteristics, improves the accuracy of advertising sorting, captures the correlation between user characteristics and advertising attributes, and improves the effectiveness of advertising recommendations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090559A_ABST
    Figure CN119090559A_ABST
Patent Text Reader

Abstract

The invention relates to an advertisement sorting method and device, electronic equipment and a storage medium, and the method comprises the steps: constructing a directed graph with a weight through user sequence behaviors in a time window, the user sequence behaviors being user behaviors of a user for advertisements, and each user behavior having a corresponding time window and a weight; carrying out random walk on the directed graph to obtain a plurality of advertisement sequences; inputting the plurality of advertisement sequences and the side information used for advertisement cold start into a to-be-trained graph embedding model to obtain a graph embedding vector, and processing the graph embedding vector into a feature vector of a fine arrangement model; wherein the feature vector comprises the ID feature of the advertisement and the historical behavior sequence feature of the user; and inputting the feature vectors into a fine ranking model to obtain a ranking result. Through the method and the device, the problem that user behavior preferences cannot be reflected due to the fact that a machine learning model is relatively single in processing of advertisement side features in the prior art is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of advertisement ranking, and in particular to an advertisement ranking method and device, an electronic device and a storage medium. Background Art

[0002] In the scenario of Internet advertising recommendation, user-side features and ID-type features from the advertising side often contain important information. However, when processing such features, current machine learning models only process advertising-side features in a simple manner, which makes it impossible to reflect user behavior preferences and thus effectively improve the click-through rate (CTR) and conversion rate (CVR) of advertisements.

[0003] There is currently no effective solution to the above problems in the prior art. Summary of the invention

[0004] The present application provides an advertisement sorting method and device, an electronic device and a storage medium to solve the problem in the prior art that the machine learning model has a relatively simple processing of advertisement side features, resulting in the inability to reflect user behavior preferences.

[0005] In a first aspect, the present application provides a method for sorting advertisements, comprising: constructing a directed graph with weights by using user sequence behaviors within a time window, wherein the user sequence behaviors refer to user behaviors towards advertisements, and each user behavior has a corresponding time window and weight; performing a random walk on the directed graph to obtain multiple advertisement sequences; inputting the multiple advertisement sequences and the side information for cold start of advertisements into a graph embedding model to be trained to obtain a graph embedding vector, and processing the graph embedding vector into a feature vector of a refined ranking model; wherein the feature vector includes an ID feature of the advertisement and a historical behavior sequence feature of the user; inputting the feature vector into the refined ranking model to obtain a sorting result, wherein the sorting result represents the result of sorting advertisements based on user behavior.

[0006] Optionally, using user sequence behavior within a time window to construct a directed graph with weights includes: obtaining the user sequence behavior and determining the time window and weight corresponding to the user sequence behavior; constructing an initial directed graph based on the user sequence behavior, the time window corresponding to the user sequence behavior, and the weight; superimposing the side information and weight of each user sequence behavior in the initial directed graph to obtain the side information and edge weight of the directed graph.

[0007] Optionally, the side information and weights of each user sequence behavior in the initial directed graph are superimposed to obtain the directed graph, including: taking the union of the side information corresponding to each user sequence behavior on each edge in the initial directed graph to obtain a first result, and summing the edge weights corresponding to each user sequence behavior on each edge to obtain a second result; and constructing the directed graph based on the first result and the second result.

[0008] Optionally, performing a random walk on the directed graph to obtain multiple advertisement sequences includes: determining a transition probability of the random walk based on edge weights of the directed graph; and performing a random walk on the directed graph based on the transition probability to obtain multiple advertisement sequences associated with user behavior.

[0009] Optionally, the graph embedding vector is processed into a feature vector of a refined ranking model, including: searching the graph embedding vector for a representation vector corresponding to an ID feature of the advertisement as the ID feature; performing a pooling operation on the graph embedding vector to obtain a historical behavior sequence feature of the user; and determining a feature vector of a refined ranking model based on the ID feature and the historical behavior sequence feature.

[0010] Optionally, the multiple advertising sequences and the side information for advertising cold start are input into a graph embedding model to be trained to obtain a graph embedding vector, including: obtaining at least one of the following side information for advertising cold start: attribute information of the advertising material, promotion target information of the advertisement, and information of the advertiser; inputting the side information for advertising cold start and the multiple advertising sequences into the graph embedding model, and training the graph embedding model to obtain the graph embedding vector.

[0011] In a second aspect, the present application provides an advertisement sorting device, comprising: a first processing module, for constructing a directed graph with weights by utilizing user sequence behaviors within a time window, wherein the user sequence behaviors refer to user behaviors towards advertisements, and each user behavior has a corresponding time window and weight; a second processing module, for performing a random walk on the directed graph to obtain a plurality of advertisement sequences; a third processing module, for inputting the plurality of advertisement sequences and the side information for advertisement cold start into a graph embedding model to be trained to obtain a graph embedding vector, and processing the graph embedding vector into a feature vector of a refined ranking model; wherein the feature vector comprises an advertisement ID feature and a user's historical behavior sequence feature; a fourth processing module, for inputting the feature vector into a refined ranking model to obtain a sorting result, wherein the sorting result represents the result of sorting advertisements based on user behaviors.

[0012] Optionally, the first processing module includes: a first processing unit, used to obtain the user sequence behavior and determine the time window and weight corresponding to the user sequence behavior; a construction unit, used to construct an initial directed graph based on the user sequence behavior, the time window and weight corresponding to the user sequence behavior; a second processing unit, used to superimpose the edge information and weight of each user sequence behavior in the initial directed graph to obtain the edge information and edge weight of the directed graph.

[0013] In a third aspect, the present application provides an electronic device comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus, wherein the processor is configured to execute the advertisement sorting method described in the first aspect of the present application.

[0014] In a fourth aspect, the present application further provides a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the advertisement ranking method described in the first aspect of the present application.

[0015] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application first uses the user sequence behavior in the time window to construct a directed graph with weights, and each edge weight in the directed graph represents the weight of each user behavior on the advertisement, and then the directed graph is randomly walked, so that multiple advertisement sequences can be obtained, and then the multiple advertisement sequences and the edge information for the advertisement cold start are input into the graph embedding model to be trained to obtain the graph embedding vector, and the graph embedding vector is processed as the feature vector of the refined ranking model, and the feature vector is input into the refined ranking model to obtain the sorting result. It can be seen that in the embodiment of the present application, when constructing the directed graph, the user's behavior is reflected in the edge weight, which avoids the problem of too sparse data and introduces richer information into the model. In addition, the user behavior sequence shares the feature vector with the advertisement ID feature, and makes fuller use of the vector obtained by the pre-training model. Therefore, the refined ranking model can better capture the correlation between user characteristics and advertisement attributes, so that a more accurate advertisement ranking result can be output, which solves the problem that the machine learning model in the prior art has a relatively simple processing of advertisement side features, resulting in the inability to reflect user behavior preferences. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0019] Figure 1 A flow chart of a method for sorting advertisements provided in an embodiment of the present application;

[0020] Figure 2 A flow chart of an advertisement characterization method based on user behavior provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the structure of an advertisement sorting device provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0024] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0025] In order to solve the problem that the machine learning model in the prior art has a relatively simple processing of advertising features, resulting in the inability to reflect user behavior preferences, the present application provides an advertisement ranking method, such as Figure 1 As shown, the steps of the method include:

[0026] Step 101: Construct a weighted directed graph using the user sequence behaviors within a time window. Here, the user sequence behaviors refer to the user behaviors towards advertisements, and each type of user behavior has a corresponding time window and weight.

[0027] In a specific example, the user behaviors towards advertisements in the embodiments of the present application may be: exposure behavior, click behavior, conversion behavior, deep conversion behavior, etc. Therefore, the user sequence behaviors may refer to the behavioral operations on multiple advertisements or products in the same behavior. Taking the exposure behavior as an example, the corresponding user sequence behavior may refer to the user's exposure sequence behavior (A, B, C), where A, B, and C may be advertisements or products, that is, the exposure order of the user to advertisements A, B, and C is first A, then B, and finally C. Different user behaviors are preset with corresponding weights and time windows. Taking the above user behaviors of exposure behavior, click behavior, conversion behavior, and deep conversion behavior as examples, exposure behavior: time window t0, weight ω0. Click behavior: time window t1, weight ω1. Conversion behavior: time window t2, weight ω2. Deep conversion behavior: time window t3, weight ω3.

[0028] In a specific example, according to the different frequencies of occurrence of various behaviors, the specified time windows may be t0 < t1 < t2 < t3; according to the different degrees of interest inclination reflected by the user behaviors in the actual business scenario, the specified weights may be ω0 < ω1 < ω2 < ω3. The relationships between the above time windows and the relationships between the weights are all for illustrative purposes, and specific settings can be made according to actual needs. Based on this, taking the exposure behavior as an example, assuming the exposure sequence of user 1 is (A, B, C) and the exposure sequence of user 2 is (B, C, F), the weights of each edge in the directed graph can be expressed as <A, B>ω0, <B, C>2ω0, <C, F>ω0. Here, A, B, and C can be represented as nodes in the directed graph, that is, the weight of the edge between node A and node B is ω0. The weight of the edge between node B and node C is 2ω0. However, for the same advertisement, there may also be the user's click on it and the user's conversion behavior on this advertisement. Therefore, if the weight of the edge between this node A and node B also needs to add the weights of other behaviors to be the final weight of this edge.

[0029] Step 102: Perform random walks on the directed graph to obtain multiple advertisement sequences.

[0030] In the specific example, each node in the directed graph has an edge weight with another connected node, so the random walk is performed based on the edge weight. For example, node 1 is connected to node 2 and node 3 respectively, and the edge weight between node 1 and node 2 is 0.3, and the edge weight between node 1 and node 3 is 0.5. When the random walk reaches node 1, the next node jumps to node 2 with a probability of 3 / 8 and jumps to node 3 with a probability of 5 / 8. Because the edge weight between node 3 and node 1 is higher than the edge weight between node 2 and node 1, if the current node is an advertisement, it means that after the current user selects the advertisement corresponding to node 1, the probability of selecting the advertisement corresponding to node 3 is greater than the probability of selecting the advertisement corresponding to node 2.

[0031] Step 103, inputting multiple advertisement sequences and side information for advertisement cold start into the graph embedding model to be trained to obtain a graph embedding vector, and processing the graph embedding vector into a feature vector of the refined ranking model; wherein the feature vector includes an advertisement ID feature and a user's historical behavior sequence feature;

[0032] It should be noted that the graph embedding model in the embodiment of the present application can be an EGES (Enhanced Graph Embedding with Side information) model, which is based on the Deep Walk graph embedding model based on random walks, and adds some side information to solve the problem of cold start. Based on this, considering the sparse data volume of advertising scenarios and the characteristics of user behavior that are difficult to be continuous, in the embodiment of the present application, in the stage of constructing a directed graph, several exposure samples are used to enrich the training data; at the same time, time windows and weights of different lengths are set for different stages of the user's exposure, click, conversion, and payment behavior, so that it is closer to the actual business meaning.

[0033] In a specific example, the side information used for advertising cold start in the embodiment of the present application may include information about the advertising material itself (such as color tone and size), information about the promotion target (such as industry classification), and information about the advertiser (such as conversion goals and billing methods).

[0034] ID (Identity Document) features refer to those features that cannot be directly expressed in numbers, such as the style of the movie, director and actor information, the ID of the movie the user has watched, the user's gender, geographic location information, the current season, time (morning, afternoon, evening), weather, etc. Although these features cannot be directly expressed in numerical values, they are also important in machine learning and data analysis. The processing of ID features usually involves converting them into numerical feature vectors to facilitate mathematical calculations and model training. For example, the user's gender, geographic location, etc. can be converted into numbers through encoding, such as male can be represented by the number 1 and female can be represented by the number 0.

[0035] Therefore, in the embodiment of the present application, the ID features include the advertisement ID features and the user's behavior sequence features. These ID features play an important role in the recommendation system. They can not only uniquely identify advertisements and users, but also provide unique information about users and advertisements, thereby helping the model to make more accurate recommendations.

[0036] Step 104: input the feature vector into a refined ranking model to obtain a ranking result, wherein the ranking result represents the result of ranking the advertisements based on user behavior.

[0037] It can be seen that in the embodiment of the present application, the refined ranking model can include a large number of ID-type features, and the embedding parameters of such features are obtained through pre-training, integrated into the bottom layer of the overall refined ranking model, participate in the upper network training, and converge to the optimal solution. The refined ranking model in the embodiment of the present application is used to recommend the most likely items or content of interest to users by analyzing factors such as user historical behavior and interest preferences.

[0038] Through the steps 101 to 104 of the above-mentioned embodiment of the present application, the user sequence behavior in the time window is first used to construct a directed graph with weights, and each edge weight in the directed graph represents the weight of each user's behavior on the advertisement, and then the directed graph is randomly walked, so that multiple advertisement sequences can be obtained, and then the multiple advertisement sequences and the edge information for the advertisement cold start are input into the graph embedding model to be trained to obtain the graph embedding vector, and the graph embedding vector is processed as the feature vector of the refined ranking model, and the feature vector is input into the refined ranking model to obtain the sorting result. It can be seen that in the embodiment of the present application, when constructing the directed graph, the user's behavior is reflected in the edge weight, which avoids the problem of too sparse data and introduces richer information into the model. In addition, the user behavior sequence shares the feature vector with the advertisement ID feature, and makes fuller use of the vector obtained by the pre-training model. Therefore, the refined ranking model can better capture the correlation between user characteristics and advertisement attributes, so that a more accurate advertisement ranking result can be output, which solves the problem that the machine learning model in the prior art has a relatively simple processing of advertisement side features, resulting in the inability to reflect user behavior preferences.

[0039] In the embodiment of the present application, the method of constructing a weighted directed graph by utilizing the user sequence behavior in the time window involved in the above step 101 may further include:

[0040] Step 11, obtaining user sequence behavior, and determining the time window and weight corresponding to the user sequence behavior;

[0041] Taking the user behaviors of users on advertisements as exposure behavior, click behavior, conversion behavior, and deep conversion behavior as an example, the user sequence behavior in the embodiment of the present application may refer to the behavioral operations on multiple advertisements or commodities in the same behavior. Taking the exposure behavior as an example, the corresponding user sequence behavior may refer to the user's exposure sequence behavior (A, B, C), where A, B, and C may be advertisements or commodities, that is, the user's exposure order for advertisements A, B, and C is A first, then B, and finally C. Different user behaviors are preset with corresponding weights and time windows. Taking the above user behaviors as exposure behavior, click behavior, conversion behavior, and deep conversion behavior as an example, exposure behavior: time window t0, weight ω0. Click behavior: time window t1, weight ω1. Conversion behavior: time window t2, weight ω2. Deep conversion behavior: time window t3, weight ω3.

[0042] Step 12, constructing an initial directed graph based on the user sequence behavior, the time window corresponding to the user sequence behavior, and the weight;

[0043] Step 13, superimpose the side information and weight of each user sequence behavior in the initial directed graph to obtain the side information and edge weight of the directed graph.

[0044] Taking exposure behavior as an example, assuming that the exposure sequence of user 1 is (A, B, C) and the exposure sequence of user 2 is (B, C, F), the directed graph can be expressed as:<A,B> ω0,<B,C> 2ω0,<C,F> ω0, where ABC can be represented as nodes in a directed graph, that is, the edge weight between node A and node B is ω0. The edge weight between node B and node C is 2ω0, so that the initial directed graph can be obtained. However, for the same advertisement, there may be behaviors such as users clicking on it and users converting to the advertisement. Therefore, if the edge weight between node A and node B needs to be added with the weight of other behaviors, it is the final edge weight of the edge. Therefore, in a specific embodiment, when the exposure, click, conversion, and deep conversion behaviors are superimposed, it is only necessary to take the union of the edge information and sum the weight information. Finally, the weight from node i to node j can be defined as:

[0045]

[0046] Among them, N is the number of users, n is the user number, and p represents a type of exposure / click / conversion / deep conversion behavior. It is the set of all sessions (user behavior sequences) of a single user within the sample period. is an indicative function, indicating whether a single user has behavior records from i to j in the current session (user behavior sequence) of the current behavior.

[0047] It can be seen that in the embodiment of the present application, a directed graph can be constructed in combination with the user behavior towards the advertisement and its corresponding weight, from which the association relationship between the user and the advertisement can be obtained, so that the embedding parameters shared by the user behavior sequence and the advertisement ID features can be obtained through the graph embedding model in the subsequent process, so that the model can better capture the semantic association information between the user characteristics and the advertisement attributes, thereby enabling the refined ranking model to more accurately sort the advertisements.

[0048] The method of obtaining a directed graph by superimposing the edge information and weights of the sequence behaviors of each user in the initial directed graph involved in step 13 may further include:

[0049] Step 21, taking the union of the edge information corresponding to each user sequence behavior on each edge in the initial directed graph to obtain a first result, and summing the edge weights corresponding to each user sequence behavior on each edge to obtain a second result;

[0050] Step 22: construct a directed graph based on the first result and the second result.

[0051] It can be seen that in the embodiment of the present application, when constructing the final directed graph required, the repeated edge information can be removed. Specifically, the edge information can include the information of the advertising material itself (such as color tone and size), the information of the promotion target (such as industry classification), and the information of the advertiser (such as conversion target and billing method), and the weights of different behaviors need to be superimposed. In this way, the edge information in the final directed graph required is richer, and the edge weight can also reflect the inclination of each behavior to the advertisement.

[0052] In the embodiment of the present application, the method of performing random walk on the directed graph involved in the above step 102 to obtain multiple advertisement sequences may further include:

[0053] Step 31, determining the transition probability of the random walk based on the edge weights of the directed graph;

[0054] Step 32: Perform a random walk on the directed graph based on the transition probability to obtain multiple advertisement sequences associated with the user behavior.

[0055] In this regard, in a specific example, each node in a directed graph has an edge weight with another connected node, so when performing a random walk, it is based on the edge weight. For example, node 1 is connected to node 2 and node 3 respectively, and the edge weight between node 1 and node 2 is 0.3, and the edge weight between node 1 and node 3 is 0.5. When performing a random walk to node 1, the next node jumps to node 2 with a probability of 3 / 8 and jumps to node 3 with a probability of 5 / 8, because the edge weight between node 3 and node 1 is higher than the edge weight between node 2 and node 1. If the current node is an advertisement, it means that after the current user selects the advertisement corresponding to node 1, the probability of selecting the advertisement corresponding to node 3 is greater than the probability of selecting the advertisement corresponding to node 2.

[0056] In an optional implementation of the embodiment of the present application, the method of processing the graph embedding vector into a feature vector of a refined ranking model involved in the above step 103 may further include:

[0057] Step 41, searching the graph embedding vector for a representation vector corresponding to the ID feature of the advertisement as the ID feature;

[0058] Step 42, performing a pooling operation on the graph embedding vector to obtain the user's historical behavior sequence features;

[0059] It should be noted that the graph embedding vector is the process of mapping graph data (usually a high-dimensional sparse matrix) into a low-dimensional dense vector. Therefore, in the embodiment of the present application, the data in the directed graph, namely the user behavior sequence features and the advertising ID features, and the corresponding weights, can be mapped into a low-order dense graph embedding vector through the graph embedding model, so that the advertising ID features and the user's historical behavior sequence features can be better obtained later.

[0060] Step 43: Determine the feature vector of the refined ranking model based on the ID feature and the historical behavior sequence feature.

[0061] It can be seen that in the embodiment of the present application, there are two types of features represented by the graph embedding vector: advertising ID features and user historical behavior sequence features. For advertising ID features, the representation vector corresponding to the ID is directly searched from the graph embedding vector as a feature; for user historical behavior sequence features, simple pooling or weighted pooling can be used to obtain them. Pooling is used to reduce the dimension of the feature graph.

[0062] It can be seen that in the embodiment of the present application, the user behavior sequence and the advertising ID share embedding parameters, which makes fuller use of the vectors obtained by the pre-trained model, and the model can better capture the semantic association information between user characteristics and advertising attributes.

[0063] In an optional implementation of the embodiment of the present application, the method of inputting multiple advertisement sequences and side information for advertisement cold start into the graph embedding model to be trained involved in the above step 103 to obtain the graph embedding vector may further include:

[0064] Step 51, obtaining at least one of the following side information for cold start of advertisement: attribute information of advertisement material, promotion target information of advertisement, and information of advertiser;

[0065] Step 52, inputting the side information for advertising cold start and multiple advertising sequences into the graph embedding model;

[0066] Step 53: Train the graph embedding model to obtain a graph embedding vector.

[0067] In the embodiment of the present application, in the advertisement recommendation scenario, side information can effectively solve the cold start problem in the initial stage of advertisement delivery.

[0068] The present application is explained below in conjunction with a specific implementation of an embodiment of the present application. The specific implementation provides an advertisement characterization method based on user behavior, such as Figure 2 As shown, the steps of the method include:

[0069] Step 201: Obtain the exposure / click / conversion / deep conversion records of the user;

[0070] Step 202: Determine the window duration and behavior weights of various behaviors, divide Sessions for the user, and construct a directed graph;

[0071] Step 203: Superimpose the edge and weight information of various behaviors to re-obtain the final directed graph;

[0072] For the above steps 201 to 203, it is possible to first construct a weighted directed graph using the user sequential behaviors within the time window, such as the exposure behavior: time window t0, weight ω0; click behavior: time window t1, weight ω1; conversion behavior: time window t2, weight ω2; deep conversion behavior: time window t3, weight ω3.

[0073] In addition, according to the different frequencies of occurrence of various behaviors, the specified time windows should satisfy t0 < t1 < t2 < t3; according to the different degrees of interest tendency reflected by user behaviors in the actual business scenario, the specified weights should satisfy ω0 < ω1 < ω2 < ω3.

[0074] Taking the exposure behavior as an example, assume the exposure sequence of user 1 is (A, B, C), and the exposure sequence of user 2 is (B, C, F), then the directed graph can be expressed as: <A, B>ω0, <B, C>2ω0, <C, F>ω0. Among them, A, B, and C can be advertisements or products.

[0075] In addition, when superimposing the exposure, click, conversion, and deep conversion behaviors, only the union of the edge information needs to be taken, and the weight information needs to be summed. Finally, the weight from node i to node j can be defined as:

[0076]

[0077] where N is the number of users, n represents the user number, p represents one of the exposure / click / conversion / deep conversion behaviors, is the set of all sessions (user behavior sequences) of a single user within the sample period, is the indicator function, indicating whether there is a behavior record from i to j for a single user in the current behavior and current session.

[0078] Step 204: Input the sequence obtained by the walk and the edge information of the advertisement cold start, train the EGES model, and output the advertisement vector representation;

[0079] In an embodiment of the present application, a random walk is performed based on a directed graph and edge weight information to generate several advertising sequences. Based on the generated directed graph, a random walk is performed, and its transition probability is calculated based on the weights in the directed graph. Then, in combination with appropriate side information, the EGES model structure is used to train it (side information and several advertising sequences) to obtain an output matrix. In the scenario of recommended advertising, side information can effectively solve the cold start problem in the early stage of advertising delivery. The side information should include information about the advertising material itself (such as color tone, size), information about the promotion target (such as industry classification), and information about the advertiser (such as conversion goals, billing methods).

[0080] By searching in the output matrix, we can get the graph embedding vector corresponding to the advertisement. There are two types of features represented by the graph embedding vector: advertisement ID features and user historical behavior sequence features. For advertisement ID features, we directly search for the representation vector corresponding to the ID in the output matrix as the feature; for user historical behavior sequence features, we can use simple pooling or weighted pooling to obtain them.

[0081] Step 205: training the sorting model and outputting the sorting result after the training is completed.

[0082] To this end, the advertising ID features and the user's historical behavior sequence features are spliced ​​into the fully linked layer of the refined ranking model to participate in the upper-layer model training, and then the ranking results are output.

[0083] It should be noted that the graph embedding model in the embodiment of the present application is the EGES model, which is based on the Deep Walk graph embedding model based on random walks, and adds some side information to solve the problem of cold start. Taking into account the sparse data volume of advertising scenarios and the characteristics of user behavior that is difficult to be continuous, in the stage of constructing a directed graph, several exposure samples are used in the embodiment of the present application to enrich the training data; at the same time, time windows and weights of different lengths are set for different stages of the user's click, conversion, and payment behavior to make it closer to the actual business meaning.

[0084] In addition, in the embodiment of the present application, the graph embedding model and the refined ranking model are separated, and the two can use different update frequencies. For the graph embedding model, since the advertising attributes often do not change frequently, the full amount of data is used for training and the update frequency is maintained at the daily level; for the refined ranking recommendation model, since the parameters also contain other user-side or cross-End2EndEmbedding parameters, these parameters are updated in real time based on the daily offline model.

[0085] It can be seen that in the embodiments of the present application, weighted superposition is performed on multiple segments of user behavior when constructing a directed graph, avoiding the problem of too sparse data and introducing richer information into the model. The adjustability of behavior weights enables it to adapt to different scenarios. The pre-trained graph embedding model and the online refined ranking model use different update frequencies, which reduces the time required for training. The user behavior sequence shares the embedding parameters with the advertising ID, making more full use of the vectors obtained by the pre-trained model, and the model can better capture the semantic association information between user characteristics and advertising attributes.

[0086] Corresponding to the above Figure 1 , the embodiment of the present application provides an advertisement ranking device, such as Figure 3 As shown, the device comprises:

[0087] The first processing module 302 is used to construct a weighted directed graph using user sequence behaviors within a time window, wherein user sequence behaviors refer to user behaviors of users on advertisements, and each user behavior has a corresponding time window and weight;

[0088] The second processing module 304 is used to perform a random walk on the directed graph to obtain multiple advertisement sequences;

[0089] The third processing module 306 is used to input the multiple advertisement sequences and the side information for advertisement cold start into the graph embedding model to be trained to obtain the graph embedding vector, and process the graph embedding vector into a feature vector of the refined ranking model; wherein the feature vector includes the ID feature of the advertisement and the historical behavior sequence feature of the user;

[0090] The fourth processing module 308 is used to input the feature vector into the refined ranking model to obtain a ranking result, wherein the ranking result represents the result of ranking the advertisements based on user behavior.

[0091] Through the device of the embodiment of the present application, the user sequence behavior in the time window is first used to construct a directed graph with weights, in which each edge weight in the directed graph represents the weight of each user behavior on the advertisement, and then the directed graph is randomly walked, so that multiple advertisement sequences can be obtained, and then the multiple advertisement sequences and the edge information for the advertisement cold start are input into the graph embedding model to be trained to obtain the graph embedding vector, and the graph embedding vector is processed as the feature vector of the refined ranking model, and the feature vector is input into the refined ranking model to obtain the sorting result. It can be seen that in the embodiment of the present application, when constructing the directed graph, the user's behavior is reflected in the edge weight, which avoids the problem of too sparse data and introduces richer information to the model. In addition, the user behavior sequence shares the feature vector with the advertisement ID feature, and makes fuller use of the vector obtained by the pre-training model. Therefore, the refined ranking model can better capture the correlation between user characteristics and advertisement attributes, so that a more accurate advertisement ranking result can be output, which solves the problem that the machine learning model in the prior art has a relatively simple processing of advertisement side features, resulting in the inability to reflect user behavior preferences.

[0092] In an optional implementation manner of an embodiment of the present application, the first processing module in the embodiment of the present application may include: a first processing unit, used to obtain user sequence behavior and determine the time window and weight corresponding to the user sequence behavior; a construction unit, used to construct an initial directed graph based on the user sequence behavior, the time window corresponding to the user sequence behavior, and the weight; a second processing unit, used to superimpose the edge information and weight of each user sequence behavior in the initial directed graph to obtain the edge information and edge weight of the directed graph.

[0093] In an optional implementation manner of an embodiment of the present application, the second processing unit in the embodiment of the present application may include: a first processing sub-unit, used to take the union of edge information corresponding to each user sequence behavior on each edge in the initial directed graph to obtain a first result, and sum up the edge weights corresponding to each user sequence behavior on each edge to obtain a second result; a second processing sub-unit, used to construct a directed graph based on the first result and the second result.

[0094] In an optional implementation of an embodiment of the present application, the second processing module in the embodiment of the present application may include: a determination unit for determining the transition probability of a random walk based on the edge weights of a directed graph; and a third processing unit for performing a random walk on the directed graph based on the transition probability to obtain multiple advertising sequences associated with user behavior.

[0095] In an optional implementation of an embodiment of the present application, the third processing module in the embodiment of the present application may include: a fourth processing unit, used to search for a representation vector corresponding to the ID feature of the advertisement from the graph embedding vector as the ID feature; a fifth processing unit, used to perform a pooling operation on the graph embedding vector to obtain the user's historical behavior sequence feature; and a sixth processing unit, used to determine the feature vector of the refined ranking model based on the ID feature and the historical behavior sequence feature.

[0096] In an optional implementation manner of an embodiment of the present application, the third processing module in the embodiment of the present application may include: an acquisition unit, used to acquire at least one of the following side information for advertising cold start: attribute information of the advertising material, promotion target information of the advertisement, and information of the advertiser; a seventh processing unit, used to input the side information for advertising cold start and multiple advertising sequences into a graph embedding model; an eighth processing unit, used to train the graph embedding model to obtain a graph embedding vector.

[0097] like Figure 4 As shown, an embodiment of the present application provides an electronic device, including a processor 411, a communication interface 412, a memory 413 and a communication bus 414, wherein the processor 411, the communication interface 412, and the memory 413 communicate with each other through the communication bus 414.

[0098] Memory 413, used for storing computer programs;

[0099] In one embodiment of the present application, the processor 411 is used to execute the program stored in the memory 413 to implement the advertisement sorting method provided by any of the aforementioned method embodiments, and the role it plays is similar and will not be repeated here.

[0100] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the advertisement sorting method provided in any of the aforementioned method embodiments are implemented.

[0101] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0103] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0104] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for ranking advertisements, characterized in that: include: Using the user sequence behavior within the time window, a directed graph with weights is constructed, wherein the user sequence behavior refers to the user behavior of the user on the advertisement, and each user behavior has a corresponding time window and weight; Performing a random walk on the directed graph to obtain multiple advertisement sequences; Input the plurality of advertisement sequences and the side information for advertisement cold start into the graph embedding model to be trained to obtain a graph embedding vector, and process the graph embedding vector into a feature vector of the refined ranking model; wherein the feature vector includes an advertisement ID feature and a user's historical behavior sequence feature; The feature vector is input into a refined ranking model to obtain a ranking result, wherein the ranking result represents a result of ranking advertisements based on user behavior.

2. The method according to claim 1, characterized in that: Using the user sequence behavior within the time window, a weighted directed graph is constructed, including: Acquire the user sequence behavior, and determine a time window and a weight corresponding to the user sequence behavior; constructing an initial directed graph based on the user sequence behavior, the time window corresponding to the user sequence behavior, and the weight; The side information and the weight of each user sequence behavior in the initial directed graph are superimposed to obtain the side information and the edge weight of the directed graph.

3. The method according to claim 2, characterized in that The directed graph is obtained by superimposing the edge information and weights of the sequence behaviors of each user in the initial directed graph, including: Taking the union of the edge information corresponding to each user sequence behavior on each edge in the initial directed graph to obtain a first result, and summing the edge weights corresponding to each user sequence behavior on each edge to obtain a second result; The directed graph is constructed based on the first result and the second result.

4. The method according to claim 3, characterized in that Performing a random walk on the directed graph to obtain multiple advertisement sequences includes: Determining a transition probability of the random walk based on edge weights of the directed graph; The directed graph is randomly walked based on the transition probability to obtain a plurality of advertisement sequences associated with the user behavior.

5. The method according to claim 1, characterized in that Processing the graph embedding vector into a feature vector of a refined ranking model includes: Searching for a representation vector corresponding to the ID feature of the advertisement from the graph embedding vector as the ID feature; Performing a pooling operation on the graph embedding vector to obtain the historical behavior sequence features of the user; A feature vector of a refined ranking model is determined based on the ID feature and the historical behavior sequence feature.

6. The method according to claim 1, characterized in that Inputting the plurality of advertisement sequences and the side information for advertisement cold start into the graph embedding model to be trained, and obtaining the graph embedding vector includes: Acquire at least one of the following side information for cold start of advertisement: attribute information of advertisement material, promotion target information of advertisement, and information of advertiser; The side information for advertising cold start and the plurality of advertisement sequences are input into the graph embedding model, and the graph embedding model is trained to obtain the graph embedding vector.

7. An advertisement sorting device, characterized in that: include: A first processing module is used to construct a directed graph with weights by using user sequence behaviors within a time window, wherein the user sequence behaviors refer to user behaviors of users on advertisements, and each user behavior has a corresponding time window and weight; A second processing module is used to perform a random walk on the directed graph to obtain multiple advertisement sequences; A third processing module is used to input the multiple advertisement sequences and the side information for advertisement cold start into the graph embedding model to be trained to obtain a graph embedding vector, and process the graph embedding vector into a feature vector of the refined ranking model; wherein the feature vector includes an ID feature of the advertisement and a historical behavior sequence feature of the user; The fourth processing module is used to input the feature vector into a refined ranking model to obtain a ranking result, wherein the ranking result represents a result of ranking advertisements based on user behavior.

8. The device according to claim 7, characterized in that The first processing module includes: A first processing unit, configured to obtain the user sequence behavior and determine a time window and a weight corresponding to the user sequence behavior; A construction unit, configured to construct an initial directed graph based on the user sequence behavior, the time window corresponding to the user sequence behavior, and the weight; The second processing unit is used to superimpose the edge information and weight of each user sequence behavior in the initial directed graph to obtain the edge information and edge weight of the directed graph.

9. An electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor coupled to the at least one bus; At least one memory connected to the at least one bus, wherein the processor is configured to execute the advertisement sorting method according to any one of claims 1 to 6.

10. A computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the advertisement ranking method according to any one of claims 1 to 6.