Method and device for pushing information

Abstract

The present invention provides a method and device for pushing information, which mainly include: a user pulls down to refresh the homepage of a client to initiate a request; the request is transmitted by the client to a server interface, and at the same time, the client also sends the user and user context features to the server interface, and the client also sends the user behavior log to a message queue; after the server receives the request from the client, it calls a feature push system interface and sends the features required by the feature push algorithm model to the feature push system; after the recommendation system receives the request from the server, it requests the recall system for recommended content; after the server receives the content sets returned by the recall system, it performs sorting processing.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a method and device for determining appropriate user feature information through data mining or big data, and in particular to a method and device for pushing information. Background Art

[0002] At present, the Internet has become the main channel for most people to obtain information, and accordingly, the amount of information on the Internet has exploded exponentially. Especially with the popularization and development of mobile devices, while the amount of information is becoming more and more abundant, users' attention is also overly dispersed, making it difficult to obtain effective information.

[0003] Search engines provide a means to solve the above problems. However, in the prior art, search engines rely on users to enter keywords for active retrieval, which results in poor user experience, especially on mobile Internet terminals. At the same time, the system cannot automatically obtain user interests and complete push notifications.

[0004] Therefore, providing users with valuable and interesting information centered on their interests has become an urgent problem to be solved. One of the more acute challenges is the cold start of the recommendation system and the continuous updating of recommended information. That is, when a platform acquires a new user, how can it recommend content of interest to the user without sufficient data on its own platform? A more common solution is to obtain information from users' various social networks in an appropriate manner, including but not limited to social accounts such as WeChat, QQ, and Alipay, and analyze the data based on the above information to obtain effective recommendations.

[0005] However, the browsing and publishing content of users' social networks is often messy and contains various information. It also includes a large number of new network terms, such as yyds and a large number of homophonic words, which are often difficult to be understood by the NLP network of existing technology. Therefore, it greatly restricts the accuracy of recommendations.

[0006] The challenge of continuously updating recommended information is that after users have accumulated a certain amount of usage behavior on the platform, although the platform itself has obtained relatively sufficient information, users' interests are often reflected in other social platforms where they have used the platform for a longer time, such as WeChat and other platforms with higher user stickiness. For some specialized platforms, if they are only based on their own data, there will inevitably be deviations from the real interests of users.

[0007] There is no good solution to the above problems in the prior art. Summary of the invention

[0008] The invention discloses a method and a device for pushing information.

[0009] The present invention discloses a device for pushing information, characterized in that: the device comprises a feature pushing system, a recall system, a sorting system, and / or a re-sorting system, the recall system obtains candidate information about a user from a material library;

[0010] The sorting system processes the information provided by the recall system;

[0011] The reordering system processes the processing result of the ordering system;

[0012] The material library includes Hailiang’s news content.

[0013] The feature push system further includes: a content data dimension label subsystem, and the data dimension label subsystem is used to process the first user data obtained from the social network software to obtain a feature description of the user.

[0014] The content data dimension label subsystem processes the first user data according to specific dimensions, where the specific dimensions include time dimension, interaction dimension, content dimension, and / or user dimension.

[0015] The time dimension further includes publishing time, last reply time, and / or last operation time.

[0016] The interaction dimension further includes reading volume, reply volume, collection volume, like volume, reply like volume, and / or sharing volume.

[0017] The content dimensions further include content length, average reply length, and / or number of pictures.

[0018] The user dimension further includes the user's interest in the content, activity, and / or the reputation of the user who posts / replies to the content.

[0019] The interaction volume dimension is determined by weighting various data. Preferably, the weight values ​​vary. The weight of reading volume is usually the lowest, and the weight values ​​of reply, like / collection, and sharing volume are relatively high, and are determined based on the business scenario.

[0020] A method for pushing information, comprising the following steps:

[0021] Step A1: The user pulls down to refresh the client homepage to initiate a request;

[0022] Step A2: The request is transmitted from the client to the server interface. At the same time, the client also sends the user and user context features to the server interface. The client also sends the user behavior log to the message queue.

[0023] Step A3: After receiving the request from the client, the server calls the feature push system interface and sends the features required by the feature push algorithm model to the feature push system;

[0024] Step A4: After receiving the request from the server, the recommendation system will request the recall system for recommended content;

[0025] Step A5: After receiving the content sets returned by the recall system, the server performs sorting processing.

[0026] The user context features include: user ID, machine model, system, request time, request IP, location, etc.;

[0027] The step A3 further comprises:

[0028] Step A3-1: Store the user behavior data in the subscription message database into the data warehouse and partition them according to specific time features;

[0029] Step A3-2: searching the large wide table of the data warehouse for daily activity characteristics of the user in the past specific time period, and storing them in a first data structure;

[0030] Step A3-3: Based on the number of user behaviors in each channel in the past 15 days, a custom UDF function is used to perform weighted cumulative calculation to obtain a weighted score;

[0031] The calculation formula of the UDF function is: interest score = ∑ number of behaviors in the past i days * a i , where a usually ranges from 0.5 to 0.99, indicating the rate at which the user’s interest in the channel content decays. After multiple tests, the optimal value of a is finally determined to be 0.95;

[0032] Step A3-4: Perform outlier processing on the weight score, wherein the outlier processing refers to assigning a value of 0 to a null weight value and retaining two decimal places to a non-null weight value.

[0033] The large wide table refers to a data table that summarizes user-side information, content-side information, and context features, and is used for preparation before training the machine learning model;

[0034] Preferably, the user-side information includes user ID, gender, age, mobile phone model, mobile phone system, recently clicked content, exposed content, shared content, favorite content, interest scores for content on different channels recorded in the portrait, etc.;

[0035] The portrait is to collect various behaviors of the user in real time through a real-time computing program to generate a detailed description of the user, which includes the user's behavioral footprint and the user's interest score for the content of each channel calculated based on the behavioral footprint;

[0036] The content side information includes content ID, content title, content type, number of clicks on the content, number of exposures of the content (exposure means appearing on the current screen of the mobile phone), number of shares of the content, number of binary scores of the content (number of secondary shares), number of collections of the content, number of comments on the content, number of clicks on the content in WeChat Moments, click-through rate of the content, rate of return of the content (number of off-site clicks divided by number of exposures), share rate of the content, binary rate of the content (ratio of secondary sharing) and average reading time of the content, etc.;

[0037] The context features include the time of the user's behavior, the channel the user is currently in, the location of the article on the current screen, the direction in which the user's finger slides, etc.;

[0038] Preferably, the step A3-4 further includes concatenating the user's features such as clicks, exposures, shares, favorites, click-through rates and share rates in the past 1, 2, 3, and 15 days into one field for storage;

[0039] Step A3-4 concatenates features to avoid dimensionality explosion;

[0040] Step A3-5: Generate daily features on the content side, specifically including: find out the number of exposures, clicks, shares, favorites and comments of the content in the past 1, 3, 5 and 7 days from the data warehouse, and calculate the comprehensive score through the second UDF function. The formula of the second UDF function is:

[0041] Step A3-5-a. First normalize the click rate, spread rate, rate of return, binary rate, and sharing rate. The normalization method is: (current value - minimum value) / (maximum value - minimum value);

[0042] Step A3-5-b. Calculate the comprehensive normalized score = (propagation rate normalized * 5) + (yield normalized * 40) + (click rate normalized * 45) + (binary rate normalized * 5) + (sharing rate normalized * 5);

[0043] Step A3-5-c. Calculate the weight score of each content = (number of exposures of the content + number of clicks + number of shares + number of clicks in WeChat Moments + two scores) / number of exposures;

[0044] Step A3-5-d. The final comprehensive score of the content = weight score * normalized score;

[0045] The features required by the feature push algorithm model include: request information, user side information, and / or article side information;

[0046] The subscription message database is preferably kafka;

[0047] Preferably, the data warehouse stored in step A3-1 is developed based on Alibaba Cloud DataWorks framework;

[0048] Preferably, the specific time feature selects hours and days as features respectively;

[0049] Preferably, the specific time period is 10-20 days; preferably, the specific time period is 15 days;

[0050] Preferably, the daily activity characteristics described in step 3-2 refer to the activity characteristics of each channel every day, and the channel refers to the classification and display of different types of content in the app in order to improve the user's reading experience, wherein each type of display is called a channel.

[0051] Preferably, the activity characteristics include the number of exposures, clicks, shares, comments and favorites;

[0052] Preferably, the fields of the first data structure include: user ID, channel ID (such as beautiful articles, health, hot spots), behavior type (such as exposure, click, share), number of corresponding behaviors in the past day, number of corresponding behaviors in the past 2 days, number of corresponding behaviors in the past 3 days, number of corresponding behaviors in the past 15 days;

[0053] The step A3 further includes real-time feature generation, further including:

[0054] Step A3-6: performing minute-level data aggregation of articles, specifically comprising: the real-time feature generation calculation framework selects the user behavior log stream from the subscription message database, aggregates the data of content exposure, clicks, sharing, collection and comments for one minute, and stores the results in the second database;

[0055] Step 3-7: Sampling to determine sample labels, specifically including: determining the label of each sample in the sampling stage, the features and labels constitute a complete sample, usually the content that the user has exposed and clicked is a positive sample, and the samples that have been exposed but not clicked are negative samples. Preferably, in the negative sample sampling step, in addition to the exposed but not clicked content as negative samples, the two contents below the last click in a request are also regarded as negative samples even if they have not been exposed. After the label is determined, the previous features are found from the database and data warehouse according to the content ID and user ID to splice into a complete sample.

[0056] Preferably, the aggregation result in step A3-6 is in the form of: content ID, behavior type (exposure, click, share, comment or favorite), number of behaviors in one minute (for example, if it is clicked 100 times in one minute, the value of this column is 100), statistical start time, statistical end time (indicating which one-minute content is counted), and the result is stored in the database;

[0057] Preferably, the database is hbase;

[0058] Preferably, the real-time feature generation computing framework selects the flink framework.

[0059] Aggregation refers to: accumulating and summing the number of user behaviors of the content within a certain period of time;

[0060] Based on the minute-level aggregated data in HBase, we can expand and calculate the values ​​of content exposure, clicks, shares, favorites, number of comments, click-through rate, and share rate in the past 1, 2, 4, 8, and 12 hours. These data will be stored in a separate table in HBase to facilitate the subsequent sampling of samples to obtain corresponding features.

[0061] The beneficial effects of the present invention are:

[0062] Based on the user portrait, we can know the user's historical clicks, and use content collaborative filtering (similar to beer and diapers) to make "watch again and again" recommendations. The portrait also contains the user's interest score for each category of content (calculated based on the user's clicks, exposure, sharing, comments, likes, and favorites), so that the user can recall the most popular content in the category that he or she is most interested in.

[0063] Through portraits, you can also find out what content users have never been exposed to, so as to explore their interests (for example, if you have always recommended sports and entertainment to users, you can now try to push technology content to test whether users are interested. This can not only expand user interests, but also solve the problem of the long tail that becomes narrower as you push more).

[0064] Dividing user portraits into short-term portraits and long-term portraits can solve the query and storage problems caused by the portraits themselves being too large. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A typical feature mining implementation DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0067] Example 1: Dynamic integration of evaluation platform and quality system

[0068] The invention discloses a method and a device for pushing information.

[0069] The present invention discloses a device for pushing information, characterized in that: the device comprises a user tag library system, a recall system, a sorting system, and / or a re-sorting system, the recall system obtains candidate information about the user from a material library;

[0070] The sorting system processes the information provided by the recall system;

[0071] The reordering system processes the processing result of the ordering system;

[0072] The material library includes Hailiang’s news content.

[0073] The user tag library system further includes: a content data dimension tag subsystem, and the data dimension tag subsystem is used to process the first user data obtained from the social networking software to obtain a feature description of the user.

[0074] The content data dimension label subsystem processes the first user data according to specific dimensions, where the specific dimensions include time dimension, interaction dimension, content dimension, and / or user dimension.

[0075] The time dimension further includes publishing time, last reply time, and / or last operation time.

[0076] The interaction dimension further includes reading volume, reply volume, collection volume, like volume, reply like volume, and / or sharing volume.

[0077] The content dimensions further include content length, average reply length, and / or number of pictures.

[0078] The user dimension further includes the user's interest in the content, activity, and / or the reputation of the user who posts / replies to the content.

[0079] The interaction volume dimension is determined by weighting various data. Preferably, the weight values ​​vary. The weight of reading volume is usually the lowest, and the weight values ​​of reply, like / collection, and sharing volume are relatively high, and are determined based on the business scenario.

[0080] Example 2: Evaluation of literary artists or literary and artistic works

[0081] A method for pushing information, characterized in that it comprises the following steps:

[0082] Step A1: The user pulls down to refresh the client homepage to initiate a request;

[0083] Step A2: The request is transmitted from the client to the server interface, and the client also sends the user context feature to the server interface;

[0084] Step A3: After receiving the request from the client, the server calls the recommendation system interface and sends the features required by the recommendation algorithm model to the recommendation system;

[0085] Step A4: After receiving the request from the server, the recommendation system obtains the user portrait and / or the offline trained user vector stored in the database according to the user ID, as well as a list of other users with similar interests to the user, and sends them to the recall system respectively;

[0086] Step A5: After receiving the content sets returned by the recall system, the server performs sorting processing;

[0087] The user context features include: user ID, machine model, system, request time, request IP, location, etc.;

[0088] The features required by the recommendation algorithm model include: request information, user side information, and / or article side information;

[0089] Preferably, the features are shown in the following table:

[0090]

[0091]

[0092]

[0093] The step A3 further comprises:

[0094] Step A3-1: Store the user behavior data in the subscription message database into the data warehouse and partition them according to specific time features;

[0095] Step A3-2: searching the data warehouse big wide table for the activity characteristics of the user on each channel every day during a specific time period in the past, and storing them in a first data structure;

[0096] Step A3-3: Based on the number of user behaviors in each category in the past 15 days, a custom UDF function is used to perform cumulative weighting calculations to obtain a weighted score;

[0097] The calculation formula of the UDF function is: agg_score = ∑ the number of behaviors in the past i days * 0.95 i ;

[0098] Step A3-4: performing outlier processing on the weight score, wherein the outlier processing refers to assigning a value of 0 to a weight value that is null, and retaining two decimal places to a weight value that is not null;

[0099] Preferably, the step A3-4 further includes concatenating the user's features such as clicks, exposures, shares, favorites, ctr and sharing rates in the past 1, 2, 3 and 15 days into one field for storage;

[0100] Step A3-4 concatenates features to avoid dimensionality explosion;

[0101] Step A3-5: Generate daily features of the content, specifically including: find out the number of exposures, clicks, shares, favorites and comments of the content in the past 1, 3, 5 and 7 days from the data warehouse, and calculate the comprehensive score through the second UDF function. The formula of the second UDF function is:

[0102] Step A3-5-a. First normalize the CTR, propagation rate, CPM, binary rate, and sharing rate. The normalization method is: (current value - minimum value) / (maximum value - minimum value);

[0103] Step A3-5-b. Calculate the comprehensive normalized score = (propagation rate normalized * 0) + (CPM normalized * 45) + (CTR normalized * 45) + (binary rate normalized * 5) + (sharing rate normalized * 5);

[0104] Step A3-5-c. Calculate the weight score of each content = (the current exposure UV of the content + click UV + share UV + off-site UV + secondary sharing UV) / exposure UV;

[0105] Step A3-5-d. The final comprehensive score of the content = weight score * normalized score;

[0106] The features required by the recommendation algorithm model include: request information, user side information, and / or article side information;

[0107] The subscription message database is preferably kafka;

[0108] Preferably, the data warehouse stored in step A3-1 is developed based on Alibaba Cloud DataWorks framework;

[0109] Preferably, the specific time feature selects hours and days as features respectively;

[0110] Preferably, the specific time period is 10-20 days; preferably, the specific time period is 15 days;

[0111] The channel refers to the classification and display of different types of content in the app in order to improve the user's reading experience. Each type of display is called a channel, for example, the news channel, beauty channel, technology channel, etc. in the app software;

[0112] Preferably, the activity characteristics include the number of exposures, clicks, shares, comments and favorites;

[0113] Preferably, the fields of the first data structure include: user ID, channel ID (such as beautiful articles, health, hot spots), behavior type (such as exposure, click, share), number of corresponding behaviors in the past day, number of corresponding behaviors in the past 2 days, number of corresponding behaviors in the past 3 days, number of corresponding behaviors in the past 15 days;

[0114] The step A3 further includes real-time feature generation, further including:

[0115] Step A3-6: performing minute-level data aggregation of articles, specifically comprising: the real-time feature generation calculation framework selects the user behavior log stream from the subscription message database, aggregates the data of content exposure, clicks, sharing, collection and comments for one minute, and stores the results in the second database;

[0116] Step 3-7: Sampling to determine sample labels, including: Determine the label, features and mark of each sample in the sampling stage sign To form a complete sample, usually the content that users have been exposed to and clicked is a positive sample, and the samples that have been exposed but not clicked are negative samples. This is also the case in our business. However, in addition to the negative samples that have been exposed but not clicked, the two contents that have been clicked in the last request are also considered negative samples even if they have not been exposed. After the label is determined, the previous features are found from HBase and the data warehouse according to the content ID and user ID to splice them into a complete sample.

[0117] Preferably, the aggregation result in step A3-6 is in the form of: content ID, behavior type (exposure, click, share, comment or favorite), number of behaviors in one minute (for example, if it is clicked 100 times in one minute, the value of this column is 100), statistical start time, statistical end time (indicating which one-minute content is counted), and the result is stored in hbase;

[0118] Preferably, the real-time feature generation computing framework selects the flink framework.

[0119] Aggregation refers to: accumulating and summing the number of user behaviors of the content within a certain period of time;

[0120] Based on the minute-level aggregated data in HBase, we can expand and calculate the values ​​of content exposure, clicks, shares, favorites, number of comments, CTR, and sharing rate in the past 1, 2, 4, 8, and 12 hours. These data will be stored in a separate table in HBase to facilitate the subsequent sampling of samples to obtain corresponding features.

[0121] Example 3: Evaluating economic or physical objects

[0122] For the case where Kafka is selected as the subscription message database and the data warehouse is developed based on the Alibaba Cloud DataWorks framework, this solution has the following preferred specific implementation method. It should be noted that this implementation method only shows a preferred implementation method of the present invention, and other similar implementation methods are within the scope of protection of the present invention.

[0123] The specific implementation is as follows.

[0124] 1. The user's behavior on the client will be reported to the Kafka topic of the open source stream processing platform through tracking points. On the one hand, the data in Kafka will be landed in the data warehouse on an hourly basis, and on the other hand, it will be subscribed and consumed by the Flink job program to generate real-time features.

[0125] 2. Daily data on users and content

[0126] (1) The user behavior data in Kafka is transferred to the data warehouse by the Alibaba Cloud DataWorks framework and partitioned by hour and day. First, the number of exposures, clicks, shares, comments, and favorites of users in each channel (tag_id) every day in the past 15 days is found from the data warehouse's large wide table. The result is as follows:

[0127]

[0128]

[0129] The fields here are: user ID, channel ID (such as beautiful articles, health, hot spots), behavior type (such as exposure, click, sharing), the number of corresponding behaviors in the past day, the number of corresponding behaviors in the past 2 days, the number of corresponding behaviors in the past 3 days,... the number of corresponding behaviors in the past 15 days.

[0130] (2) Based on the number of user behaviors in each category in the past 15 days, a custom UDF function is used to perform weighted accumulation to calculate a weighted score. The calculation formula of this UDF is: agg_score = ∑ number of behaviors in the past i days * 0.95 i

[0131] The result is of the form:

[0132]

[0133] The columns represent: user ID, channel ID, behavior type, and weight score;

[0134] Through the above calculation, we get the weight score of the user for each category of content. Then we do some light processing on some abnormal values, assign 0 to null values, retain two decimal places for non-null values, and concatenate the user's click, exposure, sharing, favorite, ctr and sharing rate in the past 1, 2, 3, ... 15 days as a field for storage. This is done to avoid dimensionality explosion. Finally, we concatenate some static attributes of the user.

[0135] Preferably, the features of the final generated user can be expressed as follows:

[0136] +------------------------------------------------------------------------------------+

[0137] |Field|Type|Comment|Field|Value

[0138] +------------------------------------------------------------------------------------+

[0139] |user_age|bigint|||Age|user|23

[0140] |user_sex|bigint|||Gender|user|2

[0141] |user_pregnant|bigint|||pregnancy|user|

[0142] |user_phone|string|||Phone model|user|iPhone

[0143] |user_os|string|||Mobile system|user|iOS

[0144] |user_conn_type|string|||Connection type|user|WIFI

[0145] |user_duration|double|||Duration|user

[0146] |user_ctr_video|double|||videoctr|user|0.15894039735099338

[0147] |user_avg_duration|double|||Average stay time|user

[0148] |user_share|bigint|||Number of user shares|user

[0149] |user_click|bigint|||Number of user clicks|user

[0150] |video_user_share|bigint|||Number of video user shares|user|4

[0151] |sv_user_frt|double|||Video completion rate|user

[0152] |video_user_srt|double|||Video sharing rate|user|0.29449495366961065

[0153] |user_srt|double|||User sharing rate|user

[0154] |user_read_nums|bigint|||User reading number|user

[0155] |video_user_click|bigint|||Number of user video clicks|user|141

[0156] |contentshow|string|||User's exposure to the channel|user

[0157] |contentshow_20006:11,contentshow_20007:27,contentshow_20008:20,contentshow_20011:1,contentshow_20013:3,contentshow_20014:2,contentshow_2001

[0158] 6:2,contentshow_20017:4,contentshow_20023:3,contentshow_20026:2,contentshow_20030:57,contentshow_20058:4,contentshow_20059:9,contentshow_2006

[0159] 6:2,contentshow_20076:3,contentshow_20079:1,contentshow_20080:1,contentshow_20089:2,contentshow_20093:5,contentshow_20102:1,contentshow_20116:1,contentshow_20144:39,contentshow_20605:1

[0160] |contentclick|string|||User clicks on the channel|user

[0161] |contentclick_20006:1,contentclick_20007:3,contentclick_20008:9,contentclick_20013:2,contentclick_20014:1,contentclick_20016:2,contentclick_20030:10,c

[0162] contentclick_20059:2,contentclick_20076:2,contentclick_20079:5,contentclick_20080:1,contentclick_20093:1,contentclick_20144:3

[0163] |share|string|||User's sharing of the channel|user|share_20008:1,share_20030:2

[0164] |ctr|string|||ctr of the user on the channel|user

[0165] |ctr_20008:0.009849,ctr_20030:0.020207,ctr_20059:0.230724,ctr_20080:0.094531 ,ctr_20094:0.206549,ctr_20102:0.094531,ctr_20104:0.045587,ctr_20116:0.206549

[0166] |sharerate|string|||User's sharing rate for the channel|user|shareRate_20008:0.003159,shareRate_20030:0.010821

[0167] |user_click_video|string|||Number of videos clicked by users|user|94

[0168] (3) Generating Heaven-Level Features of Content

[0169] Find the number of exposures, clicks, shares, favorites, and comments of the content in the past 1, 3, 5, and 7 days from the data warehouse, and calculate a comprehensive score through a UDF function. The formula is:

[0170] a. First normalize the CTR, transmission rate, CPM, binary rate, and sharing rate. The normalization method is: (current value - minimum value) / (maximum value - minimum value)

[0171] b. Calculate the comprehensive normalized score = (propagation rate normalized * 0) + (CPM normalized * 45) + (CTR normalized * 45) + (binary rate normalized * 5) + (sharing rate normalized * 5)

[0172] c. Calculate the weight score of each content = (current exposure UV + click UV + sharing UV + external UV + secondary sharing UV of the content) / exposure UV

[0173] d. The final comprehensive score of the content = weight score * normalized score

[0174] Then, through a similar processing method to the above user features and some static attributes of the content, the user features are finally generated; a preferred method is as follows:

[0175] +------------------------------------------------------------------------------------+

[0176] |Field|Type|Comment|Field|Value

[0177] +------------------------------------------------------------------------------------+

[0178] |article_id|string|Article ID|item|34021769

[0179] |title|string||Article Title|item|Golden Horizon | The Wind Blows Through the Wheat Waves

[0180] |abstract|string|||Article Description|item|On June 2, Wang Zheng, a reporter from Henan Daily Client, was in the wheat field of the Ninth Sub - farm of the Yellow River Flood Plain Farm in Xihua County. Farmers were operating large - scale combine harvesters to harvest wheat. During the "Three Summers" season, the wheat fields across the Central Plains are golden. The wheat harvest "battle" in the Yellow River Flood Plain Farm has reached its peak. In the wheat fields, the sound of machines roars and dust is flying, presenting a scene of bumper harvest.

[0181] |keywords|string|||Keywords|item|Wheat Field, June 2nd, Wheat, Harvest, Yellow River Flood Plain, Farm, 100,000

[0182] |type|bigint|||Content Type|item|1

[0183] |item_tag_id|bigint|||Channel Classification ID|item|20059

[0184] |item_status|bigint|||Article Status|item|1

[0185] |source|bigint|||Content Source|item|2

[0186] |item_publish_time|bigint|||Publication Time|item|1591138870

[0187] |update_time|bigint|||Update Time|item|1591138870

[0188] |publish_hours|bigint|||Hours Since Publication|item|3

[0189] |is_timeliness|bigint|||Is Timely - sensitive|item|3

[0190] |read_count|bigint|||Total number of reads|item|78

[0191] |repin_count|bigint|||Total number of favorites|item|7

[0192] |digg_count|bigint|||Total number of likes|item|7

[0193] |share_count|bigint|||Total number of shares|item|7

[0194] |bury_count|bigint|||Total number of dislikes|item|7

[0195] |has_image|bigint|||Is there an image?|item|1

[0196] |has_video|bigint|||Is there a video?|item|0

[0197] |has_audio|bigint|||Is there audio?|item|0

[0198] |duration|bigint|||Duration of the video or audio|item|0

[0199] |area_id|bigint|||citycode|user|843

[0200] |item_contentshow|string|Exposure PV and UV in the past 1, 3, 5, 7, 15 days|item

[0201] |pv_15d_nums:468,uv_15d_nums:213,pv_7d_nums:221,uv_7d_nums:99,pv_5d_nums: 147,uv_5d_nums:68,pv_3d_nums:83,uv_3d_nums:42,pv_1d_nums:48,uv_1d_nums:20

[0202] |item_contentclick|string|Click PV and UV in the past 1, 3, 5, 7, 15 days|item

[0203] |pv_15d_nums:63,uv_15d_nums:45,pv_7d_nums:40,uv_7d_nums:30,pv_5d_nums:24,uv_5d_nums:20,pv_3d_nums:17,uv_3d_nums:14,pv_1d_nums:4,uv_1d_nums:3

[0204] |item_share|string|Shared PV and UV in the past 1, 3, 5, 7, 15 days|item

[0205] |pv_15d_nums:63,uv_15d_nums:45,pv_7d_nums:40,uv_7d_nums:30,pv_5d_nums:24,uv_5d_nums:20,pv_3d_nums:17,uv_3d_nums:14,pv_1d_nums:4,uv_1d_nums:3

[0206] |item_favorite|string|Favorite PV and UV in the past 1, 3, 5, 7, 15 days|item

[0207] |pv_15d_nums:63,uv_15d_nums:45,pv_7d_nums:40,uv_7d_nums:30,pv_5d_nums:24,uv_5d_nums:20,pv_3d_nums:17,uv_3d_nums:14,pv_1d_nums:4,uv_1d_nums:3

[0208] |videoplay|string|The video playback PV and UV in the past 1, 3, 5, 7, 15 days|item

[0209] |pv_15d_nums:5,uv_15d_nums:3,pv_7d_nums:2,uv_7d_nums:1,pv_5d_nums:2,uv_5d_nums:1

[0210] |iscompleteplay|string|||The PV and UV of the video completed in the past 1, 3, 5, 7, 15 days|item

[0211] |postcontentcomment|string|||The PV and UV of video comments in the past 1, 3, 5, 7, 15 days|item

[0212] |pv_15d_nums:3,uv_15d_nums:3,pv_7d_nums:3,uv_7d_nums:3,pv_5d_nums:3,uv_5d_nums:3,pv_3d_nums:3,uv_3d_nums:3,pv_1d_nums:3,uv_1d_nums:3

[0213] |item_ctr|string|||Click-through rate PV and UV in the past 1, 3, 5, 7, 15 days|item

[0214] |pv_15d_nums:0.1493,uv_15d_nums:0.3004,pv_7d_nums:0.1493,uv_7d_nums:0.3004,pv_5d_nums:0.149 3,uv_5d_nums:0.3004,pv_3d_nums:0.1493,uv_3d_nums:0.3004,pv_1d_nums:0.1493,uv_1d_nums:0.3004

[0215] |share_show|string|||Shared / Exposure PV and UV in the past 1, 3, 5, 7, 15 days|item

[0216] |pv_15d_nums:0.0052,uv_15d_nums:0.0107,pv_7d_nums:0.0052,uv_7d_nums:0.0107,pv_5d_nums:0.005 2,uv_5d_nums:0.0107,pv_3d_nums:0.0052,uv_3d_nums:0.0107,pv_1d_nums:0.0052,uv_1d_nums:0.0107

[0217] |vplay_show|string|||Playback / exposure PV and UV in the past 1, 3, 5, 7, 15 days|item|

[0218] |favorite_show|string|||Favorite / exposure PV and UV in the past 1, 3, 5, 7, 15 days|item

[0219] |pv_15d_nums:0.0002,uv_15d_nums:0.0006,pv_7d_nums:0.0002,uv_7d_nums:0.0006,pv_5d_nums:0.000 2,uv_5d_nums:0.0006,pv_3d_nums:0.0002,uv_3d_nums:0.0006,pv_1d_nums:0.0002,uv_1d_nums:0.0006

[0220] |comment_show|string|||Comments / Exposure PV and UV in the past 1, 3, 5, 7, 15 days|item

[0221] |pv_15d_nums:0.0001,uv_15d_nums:0.0002,pv_7d_nums:0.0001,uv_7d_nums:0.0002,pv_5d_nums:0.000 1,uv_5d_nums:0.0002,pv_3d_nums:0.0001,uv_3d_nums:0.0002,pv_1d_nums:0.0001,uv_1d_nums:0.0002

[0222] |share_click|string|||Share / click PV and UV in the past 1, 3, 5, 7, 15 days|item

[0223] |pv_15d_nums:0.0346,uv_15d_nums:0.0356,pv_7d_nums:0.0346,uv_7d_nums:0.0356,pv_5d_nums:0.034 6,uv_5d_nums:0.0356,pv_3d_nums:0.0346,uv_3d_nums:0.0356,pv_1d_nums:0.0346,uv_1d_nums:0.0356

[0224] |vplay_click|string|||Play / click PV and UV in the past 1, 3, 5, 7, 15 days|item

[0225] |favorite_click|string|||Favorite / click PV and UV in the past 1, 3, 5, 7, 15 days|item

[0226] |pv_15d_nums:0.0017,uv_15d_nums:0.0021,pv_7d_nums:0.0017,uv_7d_nums:0.0021,pv_5d_nums:0.001 7,uv_5d_nums:0.0021,pv_3d_nums:0.0017,uv_3d_nums:0.0021,pv_1d_nums:0.0017,uv_1d_nums:0.0021

[0227] |comment_click|string|||Comment / click PV and UV in the past 1, 3, 5, 7, 15 days|item

[0228] |pv_15d_nums:0.0006,uv_15d_nums:0.0008,pv_7d_nums:0.0006,uv_7d_nums:0.0008,pv_5d_nums:0.000 6,uv_5d_nums:0.0008,pv_3d_nums:0.0006,uv_3d_nums:0.0008,pv_1d_nums:0.0006,uv_1d_nums:0.0008

[0229] |svplay_accomplish_ratio|string|||Playback completion rate|item

[0230] |label|bigint|||sample label|item|0

[0231] 3. Real-time feature generation

[0232] (1) Computing framework: Flink is used as the real-time computing framework;

[0233] The reason for preferring the flink framework is that although there are real-time frameworks such as storm, spark and flink, storm can only process streaming data and has no batch processing capabilities, and flink provides many advanced APIs. For example, flink's DataStream provides APIs such as Map, GroupBy, Window and Join to replace the functions of storm's bolt in one or more readers and collectors, while Storm requires programmers to implement these functions themselves; compared with spark, flink has the advantages of high throughput and low latency. In addition, flink also supports millisecond-level computing, while spark only supports second-level computing. Flink belongs to true streaming computing;

[0234] (2) Code logic: First, perform data aggregation on the article at the minute level. Based on the Flink real-time computing framework, the user behavior log stream stored in Kafka is used as the data input of the Flink real-time computing framework. Based on the user behavior log stored in Kafka, the Flink real-time computing framework first aggregates the content exposure, click, share, favorite, and comment data for one minute. The aggregation result is in the form of: content ID, behavior type (exposure, click, share, comment, or favorite), number of behaviors in one minute (for example, if it is clicked 100 times in one minute, the value of this column is 100), statistical start time, and statistical end time (indicating which one-minute content is being counted). The result is stored in HBase.

[0235] Based on the minute-level aggregated data in HBase, we can expand and calculate the values ​​of content exposure, clicks, shares, favorites, number of comments, CTR, and sharing rate in the past 1, 2, 4, 8, and 12 hours. These data will be stored in a separate table in HBase to facilitate the subsequent sampling to obtain the corresponding features.

[0236] The reasons for choosing HBase are: commonly used databases include HBase, Hive, Redis and MySQL, etc. MySQL is a relational database, but the business data involved in the present invention is large, linear expansion is required, automated operation and maintenance is required, and the application mode is simple, which is obviously not met by MySQL; although Redis is similar to HBase in function, both are NoSQL databases based on Key and Value, but Redis is more suitable for caching, and the business involved in the present invention requires that data cannot be lost, and Redis cannot guarantee this requirement. In addition, the cost of using Redis is much higher than that of HBase; Hive is a data warehouse tool with MapReduce as the underlying layer, and cannot be used for user interactive storage; Hbase has the characteristics of huge capacity, column storage, strong scalability and high reliability. In addition, the main reason why Hbase can provide real-time computing services is determined by its architecture and the underlying data structure, that is, it is determined by LSM-Tree+HTable+Cache. Compared with other database systems, the technical architecture has obvious advantages; therefore, the client can directly locate the HRegion server where the data to be queried is located, and then directly search for the data to be matched on a region of the server, and some of these data are cached.

[0237] In addition, compared with databases such as MySQL and Redis, the storage cost of HBase is also more economical. For example, the monthly cost of a core node of an HBase cluster with 8 cores, 32GB of memory, and 1800GB of disk capacity on Alibaba Cloud is only about 1000 yuan. For MySQL with the same configuration, it is 3000 yuan, and the caching database Redis is even more expensive.

[0238] (3) Sampling: In the sampling stage, the label of each sample is determined. A complete sample consists of features and a label. Usually, the content that a user has exposed and clicked on is a positive sample, and the sample that has been exposed but not clicked is used as a negative sample. This is also the case in our business. However, in addition to the samples that have been exposed but not clicked being used as negative samples, in a single request, the two pieces of content below the last click, even if they have not been exposed, are also used as negative samples. After the label is determined, the previous features are retrieved from HBase and the data warehouse based on the content ID and user ID and concatenated into a complete sample. The final features are as follows:

[0239] +------------------------------------------------------------------------------------+

[0240] |Field|Type|Comment|Field|Value

[0241] +------------------------------------------------------------------------------------+

[0242] |article_id|string|Article ID|item|34021769

[0243] |title|string||Article title|item|Golden Vision | The Wind Blows Through the Wheat Fields

[0244] |abstract|string|||Article description|item|On June 2, Wang Zheng, a reporter from the Henan Daily Client, was in the wheat fields of the Ninth Branch of the Huangfanqu Farm in Xihua County. Farmers were operating large harvesters to harvest wheat. During the "Three Summers" season, the wheat fields across the Central Plains are golden, and the wheat harvest "battle" in the Huangfanqu Farm has entered its peak. In the wheat fields, the machines are roaring and the dust is flying, presenting a scene of bumper harvest.

[0245] |keywords|string|||Keywords|item|Wheat fields, June 2, Wheat, Harvest, Huangfanqu, Farm, 100,000

[0246] |type|bigint|||Content type|item|1

[0247] |item_tag_id|bigint|||Channel category ID|item|20059

[0248] |item_status|bigint|||Article status|item|1

[0249] |source|bigint|||Content source|item|2

[0250] |item_publish_time|bigint|||Publish time|item|1591138870

[0251] |update_time|bigint|||Update time|item|1591138870

[0252] |publish_hours|bigint|||The number of hours from the publishing time to now|item|3

[0253] |is_timeliness|bigint|||Is it timeliness|item|3

[0254] |read_count|bigint|||Total number of reads|item|78

[0255] |repin_count|bigint|||Total number of favorites|item|7

[0256] |digg_count|bigint|||Total number of likes|item|7

[0257] |share_count|bigint|||Total number of shares|item|7

[0258] |bury_count|bigint|||Total number of dislikes|item|7

[0259] |has_image|bigint|||Is there an image?|item|1

[0260] |has_video|bigint|||Is there a video?|item|0

[0261] |has_audio|bigint|||Is there audio?|item|0

[0262] |duration|bigint|||Duration of the video or audio|item|0

[0263] |area_id|bigint|||citycode|user|843

[0264] |item_contentshow|string|Exposure PV and UV in the past 1, 3, 5, 7, 15 days|item

[0265] |pv_15d_nums:468,uv_15d_nums:213,pv_7d_nums:221,uv_7d_nums:99,pv_5d_nums: 147,uv_5d_nums:68,pv_3d_nums:83,uv_3d_nums:42,pv_1d_nums:48,uv_1d_nums:20

[0266] |item_contentclick|string|Click PV and UV in the past 1, 3, 5, 7, 15 days|item

[0267] |pv_15d_nums:63,uv_15d_nums:45,pv_7d_nums:40,uv_7d_nums:30,pv_5d_nums:24,uv_5d_nums:20,pv_3d_nums:17,uv_3d_nums:14,pv_1d_nums:4,uv_1d_nums:3

[0268] |item_share|string|Shared PV and UV in the past 1, 3, 5, 7, 15 days|item

[0269] |pv_15d_nums:63,uv_15d_nums:45,pv_7d_nums:40,uv_7d_nums:30,pv_5d_nums:24,uv_5d_nums:20,pv_3d_nums:17,uv_3d_nums:14,pv_1d_nums:4,uv_1d_nums:3

[0270] |item_favorite|string|Favorite PV and UV in the past 1, 3, 5, 7, 15 days|item

[0271] |pv_15d_nums:63,uv_15d_nums:45,pv_7d_nums:40,uv_7d_nums:30,pv_5d_nums:24,uv_5d_nums:20,pv_3d_nums:17,uv_3d_nums:14,pv_1d_nums:4,uv_1d_nums:3

[0272] |videoplay|string|The video playback PV and UV in the past 1, 3, 5, 7, 15 days|item

[0273] |pv_15d_nums:5,uv_15d_nums:3,pv_7d_nums:2,uv_7d_nums:1,pv_5d_nums:2,uv_5d_nums:1

[0274] |iscompleteplay|string|||The PV and UV of the video completed in the past 1, 3, 5, 7, 15 days|item

[0275] |postcontentcomment|string|||The PV and UV of video comments in the past 1, 3, 5, 7, 15 days|item

[0276] |pv_15d_nums:3,uv_15d_nums:3,pv_7d_nums:3,uv_7d_nums:3,pv_5d_nums:3,uv_5d_nums:3,pv_3d_nums:3,uv_3d_nums:3,pv_1d_nums:3,uv_1d_nums:3

[0277] |item_ctr|string|||Click-through rate PV and UV in the past 1, 3, 5, 7, 15 days|item

[0278] |pv_15d_nums:0.1493,uv_15d_nums:0.3004,pv_7d_nums:0.1493,uv_7d_nums:0.3004,pv_5d_nums:0.149 3,uv_5d_nums:0.3004,pv_3d_nums:0.1493,uv_3d_nums:0.3004,pv_1d_nums:0.1493,uv_1d_nums:0.3004

[0279] |share_show|string|||Shared / Exposure PV and UV in the past 1, 3, 5, 7, 15 days|item

[0280] |pv_15d_nums:0.0052,uv_15d_nums:0.0107,pv_7d_nums:0.0052,uv_7d_nums:0.0107,pv_5d_nums:0.005 2,uv_5d_nums:0.0107,pv_3d_nums:0.0052,uv_3d_nums:0.0107,pv_1d_nums:0.0052,uv_1d_nums:0.0107

[0281] |vplay_show|string|||Playback / exposure PV and UV in the past 1, 3, 5, 7, 15 days|item|

[0282] |favorite_show|string|||Favorite / exposure PV and UV in the past 1, 3, 5, 7, 15 days|item

[0283] |pv_15d_nums:0.0002,uv_15d_nums:0.0006,pv_7d_nums:0.0002,uv_7d_nums:0.0006,pv_5d_nums:0.000 2,uv_5d_nums:0.0006,pv_3d_nums:0.0002,uv_3d_nums:0.0006,pv_1d_nums:0.0002,uv_1d_nums:0.0006

[0284] |comment_show|string|||Comments / Exposure PV and UV in the past 1, 3, 5, 7, 15 days|item

[0285] |pv_15d_nums:0.0001,uv_15d_nums:0.0002,pv_7d_nums:0.0001,uv_7d_nums:0.0002,pv_5d_nums:0.000 1,uv_5d_nums:0.0002,pv_3d_nums:0.0001,uv_3d_nums:0.0002,pv_1d_nums:0.0001,uv_1d_nums:0.0002

[0286] |share_click|string|||Share / click PV and UV in the past 1, 3, 5, 7, 15 days|item

[0287] |pv_15d_nums:0.0346,uv_15d_nums:0.0356,pv_7d_nums:0.0346,uv_7d_nums:0.0356,pv_5d_nums:0.034 6,uv_5d_nums:0.0356,pv_3d_nums:0.0346,uv_3d_nums:0.0356,pv_1d_nums:0.0346,uv_1d_nums:0.0356

[0288] |vplay_click|string|||Play / click PV and UV in the past 1, 3, 5, 7, 15 days|item

[0289] |favorite_click|string|||Favorite / click PV and UV in the past 1, 3, 5, 7, 15 days|item

[0290] |pv_15d_nums:0.0017,uv_15d_nums:0.0021,pv_7d_nums:0.0017,uv_7d_nums:0.0021,pv_5d_nums:0.001 7,uv_5d_nums:0.0021,pv_3d_nums:0.0017,uv_3d_nums:0.0021,pv_1d_nums:0.0017,uv_1d_nums:0.0021

[0291] |comment_click|string|||Comment / click PV and UV in the past 1, 3, 5, 7, 15 days|item

[0292] |pv_15d_nums:0.0006,uv_15d_nums:0.0008,pv_7d_nums:0.0006,uv_7d_nums:0.0008,pv_5d_nums:0.000 6,uv_5d_nums:0.0008,pv_3d_nums:0.0006,uv_3d_nums:0.0008,pv_1d_nums:0.0006,uv_1d_nums:0.0008

[0293] |svplay_accomplish_ratio|string|||Playback completion rate|item

[0294] |label|bigint|||sample label|item|0

[0295] |uid|string|||user ID|user|25311456

[0296] |requestid|string|||Request ID|user|501359071604462886836102

[0297] |requesttime|string|||Request time|user|1604462886

[0298] |showlist|string|||Exposure unclicked sequence|user|33732005,33783349,33837299,33838753,33912723,33928278

[0299] |usersequence|string|||click sequence|user|33732005

[0300] |workday|string|||workday|user|1

[0301] |user_age|bigint|||Age|user|23

[0302] |user_sex|bigint|||Gender|user|2

[0303] |user_pregnant|bigint|||pregnancy|user|

[0304] |user_phone|string|||Phone model|user|iPhone

[0305] |user_os|string|||Mobile system|user|iOS

[0306] |user_conn_type|string|||Connection type|user|WIFI

[0307] |user_duration|double|||Duration|user

[0308] |user_ctr_video|double|||videoctr|user|0.15894039735099338

[0309] |user_avg_duration|double|||Average stay time|user

[0310] |user_share|bigint|||Number of user shares|user

[0311] |user_click|bigint|||Number of user clicks|user

[0312] |video_user_share|bigint|||Number of video user shares|user|4

[0313] |sv_user_frt|double|||Video completion rate|user

[0314] |video_user_srt|double|||Video sharing rate|user|0.29449495366961065

[0315] |user_srt|double|||User sharing rate|user

[0316] |user_read_nums|bigint|||User reading number|user

[0317] |video_user_click|bigint|||Number of user video clicks|user|141

[0318] |contentshow|string|||User's exposure to the channel|user

[0319] |contentshow_20006:11,contentshow_20007:27,contentshow_20008:20,contentshow_20011:1,contentshow_20013:3,contentshow_20014:2,contentshow_2001

[0320] 6:2,contentshow_20017:4,contentshow_20023:3,contentshow_20026:2,contentshow_20030:57,contentshow_20058:4,contentshow_20059:9,contentshow_2006

[0321] 6:2,contentshow_20076:3,contentshow_20079:1,contentshow_20080:1,contentshow_20089:2,contentshow_20093:5,contentshow_20102:1,contentshow_20116:1,contentshow_20144:39,contentshow_20605:1

[0322] |contentclick|string|||User clicks on the channel|user

[0323] |contentclick_20006:1,contentclick_20007:3,contentclick_20008:9,contentclick_20013:2,contentclick_20014:1,contentclick_20016:2,contentcli ck_20030:10,contentclick_20059:2,contentclick_20076:2,contentclick_20079:5,contentclick_20080:1,contentclick_20093:1,contentclick_20144:3

[0324] |share|string|||User's sharing of the channel|user|share_20008:1,share_20030:2

[0325] |ctr|string|||ctr of the user on the channel|user

[0326] |ctr_20008:0.009849,ctr_20030:0.020207,ctr_20059:0.230724,ctr_20080:0.094531 ,ctr_20094:0.206549,ctr_20102:0.094531,ctr_20104:0.045587,ctr_20116:0.206549

[0327] |sharerate|string|||User's sharing rate for the channel|user|shareRate_20008:0.003159,shareRate_20030:0.010821

[0328] |user_click_video|string|||Number of videos clicked by users|user|94

[0329] |eventtype|string|||event type|user|contentclick

[0330] |user_show_video|string|||Number of videos exposed by the user|user|806

[0331] |id|string|||item_id|item|3746

[0332] |retrieveid|string|||Distinguish scene ID|item|article_relate

[0333] |item_title_len|bigint|||Article title length|item|6

[0334] |item_content_len|bigint|||Article content length|item|56

[0335] |click_offset|bigint|||Click position|item|12

[0336] |item_type|bigint|||media type|item|2

[0337] |item_elapse_time|bigint|||Article release time from current time|item|16

[0338] |item_muid|bigint|||Media ID|item|2

[0339] |item_ty|bigint|||Media type|item|20010

[0340] |publish_time|bigint|||Publish time|item|1593590495

[0341] |tag_id|bigint|||Channel ID|item|20010

[0342] |is_timely|bigint|||Is it effective|item|0

[0343] |item_img_count_show|bigint|||Number of article images|item|3

[0344] |category|bigint|||Channel ID|item|20010

[0345] |status|bigint|||Article status|item|1

[0346] |is_up|double|||Is the item removed from shelves|

[0347] |is_hot|double|||Is it hot|item

[0348] |is_original|double|||Is it original|item

[0349] |video_click|double|||Number of video clicks|item|37

[0350] |video_share|double|||Number of video shares|item|7

[0351] |video_srt|double|||Video sharing rate|item|0.09347644411829928

[0352] |article_click|double|||Number of article clicks|item|37

[0353] |article_ctr|double|||articlectr|item|0.20833333333333334

[0354] |article_show|double|||Article exposure|item|92

[0355] |user_tag_ctr|double|||user's ctr for the channel|item

[0356] |user_tag_show|double|||User's exposure to the channel|item

[0357] |user_tag_click|double|||User clicks on the channel|item

[0358] |comment_count|double|||Number of comments|item

[0359] |source_read_count|double|||Source reading count|item

[0360] |wxsharetwice_hour|double|||hour off-site double score|item

[0361] |contentshow_hour|double|||Hourly exposure|item|1

[0362] |share_hour|double|||Number of shares per hour|item

[0363] |shareback_hour|double|||Number of clicks shared outside the site in the past hour|item

[0364] |wxcontentclick_hour|double|||Hourly off-site clicks|item

[0365] |exitcontentdetail_hour|double|||hours of stay|item|2357

[0366] |contentclick_hour|double|||Hourly clicks|item|1

[0367] |favorite_hour|double|||Favorite count per hour|item

[0368] |ctr_hour|double|||ctr for hour|item|0.063225105

[0369] |postcontentcomment_hour|double|||Number of comments per hour|item

[0370] |contentclick_12hour|double|||12 hours click count|item|12

[0371] |contentshow_12hour|double|||12 hours exposure number|item|

[0372] |ctr_12hour|double|||12 hoursctr|item|0.09453121

[0373] |exitcontentdetail_12hour|double|||12 hours stay duration|item

[0374] |wxcontentclick_12hour|double|||12 hours off-site click count|item

[0375] |wxsharetwice_12hour|double|||Number of external shares in 12 hours|item

[0376] |share_12hour|double|||Number of shares in 12 hours|item

[0377] |favorite_12hour|double|||Favorites in 12 hours|item

[0378] |postcontentcomment_12hour|double|||Number of comments in 12 hours|item

[0379] |shareback_12hour|double|||Number of clicks shared in the past 12 hours|item

[0380] |sex|bigint|||gender|user|1

[0381] |age|string|||age|user|over 45 years old

[0382] |province|string|||province|user|Shandong Province

[0383] |city|string|||city|user|Jinan

[0384] |score|bigint|||score|user|64542

[0385] |reg_days|bigint|||Registration days|user|699

[0386] |active_days|bigint|||Active days|user|698

[0387] |flag|bigint|||Whether a third party or mobile phone number has logged in|user|1

[0388] |last_login_time|bigint|||Last login time|user|1604332800

[0389] |last_clear_time|bigint|||The time of the last clear notification|user|0

[0390] |last_sys_time|bigint|||The last time the system notification was read|user|1544079479

[0391] |last_rep_time|bigint|||The last time to read the comment reply|user|1604383947

[0392] |last_login_type|bigint|||Last login platform|user|2(Last login platform, 1:ios; 2:android)

[0393] |reg_type|bigint|||Registration type|user|2(Registration platform, 1:ios; 2:android; 3:pc)

[0394] |source_id|string|||Source ID|user|c1006

[0395] |device_type|string|||Device type|user|OPPO PBBT00

[0396] |hist_share_num|bigint|||Time period sharing|user|4374

[0397] |reg_ip|bigint|||Register IP|user|1894108192

[0398] |hist_show|string|||15-day exposure categories and scores|user|

[0399] cs_20007:1551.01,cs_20086:0.54,cs_20087:34.97,cs_20097:3.41,cs_20108:29 .93,cs_20128:0.95,cs_20144:68.29,cs_20006:328.61,cs_20058:118.86,cs_2006

[0400] 9:14.37,cs_20080:192.07,cs_20093:64.22,cs_20110:5.04,cs_20604:2.44,cs_20 017:15.14,cs_20056:15.76,cs_20059:244.21,cs_20066:202.37,cs_20102:171.87

[0401] ,cs_20104:23.07,cs_20109:65.21,cs_20605:5.78,cs_20011:237.72,cs_20014:139.36,cs_20016:162.04,cs_20021:2.81,cs_20022:13.16,cs_20023:89.82,cs_2002

[0402] 6:135.82,cs_20030:3524.74,cs_20076:28.03,cs_20083:16.63,cs_20094:9.97,cs_20095:1.08,cs_20116:13.88,cs_20603:8.37,cs_20008:3105.84,cs_20010:25.01,cs_20013:80.84,cs_20015:362.75,cs_20078:2.56,cs_20079:210.8,cs_20103:4.25,cs_20589:0.57

[0403] |hist_click|string|||Category and score of clicks in 15 days

[0404] |cc_20017:1.55,cc_20056:1.03,cc_20059:44.27,cc_20066:28.14,cc_20102:90.49,cc_20104:1.61,cc_20109:27.78,cc_20006:29.73,cc_20058:9.24,cc_20080:65.87,cc_20093:34.91,cc_20110:0.86,cc_20007:263.99,cc_20087:15.68,cc_20108:21.55,cc_20144:31.77,cc_20011:69.81,cc_20014:5.46,cc_20016:26.26,cc_20022:0.63,cc_20023:2.83,cc_20026:15.17,cc_20030:353.16,cc_20083:6.67,cc_20094:7.22,cc_20116:4.25,cc_20603:3.1,cc_20008:564.64,cc_20010:2.71,cc_20013:2.42,cc_20015:28.1,cc_20078:1.99,cc_20079:69.84,cc_20103:6.29

[0405] |hist_share|string|||15-day sharing categories and scores

[0406] |ss_20011:10.2,ss_20016:3.01,ss_20026:4.54,ss_20030:75.36,ss_20094: 0.95,ss_20603:0.77,ss_20008:141.08,ss_20015:0.49,ss_20079:12.57,ss_ 20059:9.26,ss_20066:3.95,ss_20102:6.08,ss_20109:2.1,ss_20007:60.41, ss_20087:0.57,ss_20144:4.61,ss_20058:1.3,ss_20080:6.34,ss_20093:5.14

[0407] |hist_favorite|string|||15-day favorite categories and scores|fs_20102:1.63,fs_20079:0.54

[0408] |total_score|string|||15-day category total score|user

[0409] |ts_20017:1.55,ts_20056:1.03,ts_20059:72.05000000000001,ts_20066:39.99,ts_20102:121.76999999999998,ts_20104:1.61,ts_20109:34.08,ts_20006:29.73,ts_20058:13.14,ts_20080:84.89,ts_20093:50.33,ts_20110:0.86,ts_20007:445.22,ts_20087:17.39,ts_20108:21.55,ts_20144:45.6,ts_20008:987.88,ts_20010:2.71,ts_20013:2.42,ts_20015:29.57,ts_20078:1.99,ts_20079:111.87,ts_20103:6.29,ts_20011:100.41,ts_20014:5.46,ts_20016:35.29,ts_20022:0.63,ts_20023:2.83,ts_20026:28.79,ts_20030:579.24,ts_20083:6.67,ts_20094:10.07,ts_20116:4.25,ts_20603:5.41

[0410] |hist_ctr|string|||7-day category click-through rate: click / show|user

[0411] |7d_ctr_20007:0.25,7d_ctr_20008:0.2,7d_ctr_20014:0.08,7d_ctr_20030:0.23,7d_ctr_20013:0.29,7d_ctr_20066:0.25,7d_ctr_20016:0.04,7d_ctr_20109:0.5,7d_ctr_20006:0.17,7d_ctr_20011:0.33,7d_ctr_20026:0.33,7d_ctr_20058:0.18,7d_ctr_20059:0.25

[0412] |hist_sharerate|string|||7-day category favorite rate: share / click|7d_ctr_20007:0.25,7d_ctr_20008:0.2,7d_ctr_20011:0.33,7d_ctr_20058:0.18

[0413] |up_score|string|||Offline preference + real-time preference score|user

[0414] |20030:54.736000,20008:45.776000,20007:32.544000,20011:17.100000,20014:4.884000,20058:4.332000,20013:3.504000,20016:3.156000,20015:3.000000,20066:2.568000

[0415] |prefer_category_score|double|||User preference feature|user|203.614

[0416] |prefer_category_hist_ctr|double|||User preference feature|user|0.22

[0417] |prefer_hist_sharerate|double|||User preference feature|user|0.46

[0418] |prefer_hist_share|double|||User preference feature|user|0.66121438629008

[0419] |pv_15d_nums|double|||15 days reading volume|3.0|item

[0420] |pv_7d_nums|double|||7 days reading volume|2.0|item

[0421] |pv_5d_nums|double|||5 days reading volume|1.0|item

[0422] |pv_3d_nums|double|||3 days of reading|1.0|item

[0423] |pv_1d_nums|double|||1 day reading volume||item

[0424] |pv_15d_nums_click|double|||15 days pv / click|1.0|item

[0425] |pv_7d_nums_click|double|||7 days pv / click|1.0|item

[0426] |pv_5d_nums_click|double|||5 days pv / click|1.0|item

[0427] |pv_3d_nums_click|double|||3 days pv / click|1.0|item

[0428] |pv_1d_nums_click|double|||1 day pc / click|item

[0429] |pv_15d_nums_show|double|||15 days pv / show|3.0|item

[0430] |pv_7d_nums_show|double|||7 days pv / show|3.0|item

[0431] |pv_5d_nums_show|double|||5 days pv / show|3.0|item

[0432] |pv_3d_nums_show|double|||3 days pv / show|3.0|item

[0433] |pv_1d_nums_show|double|||1 day pv / show

[0434] |retrieve_type|string|||retrieve type|item|article_relate

[0435] |recall_score|double|||Recall score|item|0.04230078826

[0436] |item_ctr_15d|double|||15-day ctr of the article|0.038834951456310676|item

[0437] |item_ctr_7d|double|||Article 7-day ctr|0.038834951456310676|item

[0438] |item_ctr_5d|double|||Article 5-day ctr|0.038834951456310676|item

[0439] |item_ctr_3d|double|||3-day ctr of the article|0.038834951456310676|item

[0440] |item_ctr_1d|double|||Article one-day ctr|0.0|item

[0441] |usercategories|string|||User categories|user

[0442] |contentchannel|string|||Content column|item|Recommendation

[0443] |showtag|string|||Show tag

[0444] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for pushing information, characterized in that: The following steps are involved: Step A1: The user pulls down to refresh the client homepage to initiate a request; Step A2: The request is transmitted from the client to the server interface. At the same time, the client also sends the user and user context features to the server interface. The client also sends the user behavior log to the message queue. Step A3: After receiving the request from the client, the server calls the feature push system interface and sends the features required by the feature push algorithm model to the feature push system; Step A4: After receiving the request from the server, the recommendation system will request the recall system for recommended content; Step A5: After receiving the content sets returned by the recall system, the server performs sorting processing; The user context features include: user ID, machine model, system, request time, request IP, and location; Wherein, the step A3 further comprises: Step A3-1: Store the user behavior data in the subscription message database into the data warehouse and partition it according to specific time features; Step A3-2: searching the large wide table of the data warehouse for daily activity characteristics of the user in the past specific time period, and storing them in a first data structure; Step A3-3: Based on the number of user behaviors in each channel in the past 15 days, a custom UDF function is used to perform weighted cumulative calculation to obtain a weighted score; Step A3-4: performing outlier processing on the weight score, wherein the outlier processing refers to assigning a value of 0 to a weight value that is null, and retaining two decimal places to a weight value that is not null; The calculation formula of the UDF function is: interest score = ∑ number of behaviors in the past i days * a i , where a ranges from 0.5 to 0.99, indicating the decay rate of user interest in the channel content; Wherein, the step A3 further includes step A3-5, which specifically includes: Generating daily features on the content side specifically includes: finding out the number of exposures, clicks, shares, favorites, and comments of the content in the past 1, 3, 5, and 7 days from the data warehouse, and calculating a comprehensive score through a second UDF function. The formula of the second UDF function is: Step A3-5-a. First normalize the click rate, spread rate, rate of return, binary rate, and sharing rate. The normalization method is: (current value - minimum value) / (maximum value - minimum value); Step A3-5-b. Calculate the comprehensive normalized score = (normalized spread rate * 5) + (normalized rate of return * 40) + (normalized click rate * 45) + (normalized binary rate * 5) + (normalized sharing rate * 5); Step A3-5-c. Calculate the weight score of each content = (number of exposures of the content + number of clicks + number of shares + number of clicks in WeChat Moments + binary score) / number of exposures; Step A3-5-d. The final comprehensive score of the content = weight score * normalized score; Wherein, the step A3 further includes real-time feature generation, further including: Step A3-6: performing minute-level data aggregation of articles, specifically comprising: the real-time feature generation calculation framework selects the user behavior log stream from the subscription message database, aggregates the exposure, click, share, favorite and comment data of the content for one minute, and stores the results in the second database; Step 3-7: Sampling to determine sample labels, specifically including: determining the label of each sample in the sampling stage, the features and labels constitute a complete sample, the content that the user has exposed and clicked is the positive sample, and the samples that have been exposed but not clicked are the negative samples; preferably, in the negative sample sampling step, in addition to the exposed but not clicked samples as negative samples, the two contents below the last click in a request are also regarded as negative samples even if they have not been exposed; after the label is determined, the previous features are found from the database and data warehouse according to the content ID and user ID to splice them into a complete sample.

2. The method according to claim 1, characterized in that: The large wide table refers to a data table that summarizes user-side information, content-side information, and contextual features, and is used for preparation before training the machine learning model.

3. The method according to claim 1, characterized in that: The result of the aggregation in step A3-6 is in the form of: content ID, behavior type, number of behaviors in one minute, statistical start time, and statistical end time. The result is stored in the database.

4. The method according to claim 1, characterized in that: The database is hbase; and based on the minute-level aggregated data in hbase, the values ​​of exposure, clicks, shares, favorites, comments, click-through rate and share rate of the content in the past 1, 2, 4, 8, and 12 hours can be expanded and calculated.

5. A device for pushing information using the method of claim 1, characterized in that: The device includes a feature push system, a recall system, a sorting system, and / or a re-sorting system, wherein the recall system obtains candidate information about the user from a material library; The sorting system processes the information provided by the recall system; The reordering system processes the processing result of the ordering system; The material library includes a large amount of news content; The feature push system further includes: a content data dimension label subsystem, the data dimension label subsystem is used to process the first user data obtained from the social network software to obtain a feature description of the user; The content data dimension label subsystem processes the first user data according to a specific dimension, where the specific dimension includes a time dimension, an interaction dimension, a content dimension, and / or a user dimension; The time dimension further includes publishing time, last reply time, and / or last operation time; The interaction dimension further includes reading volume, reply volume, collection volume, like volume, reply like volume, and / or sharing volume; The content dimensions further include content length, average reply length, and / or number of pictures; The user dimension further includes the user's interest in the content, activity, and / or the reputation of the user who posted / replied to the content; The interaction dimension is determined by weighting various data.

Images