Method, apparatus, electronic device, and storage medium for determining recommendation information

Abstract

An embodiment of the present application provides a method for determining recommended information, which relates to the technical field of information recommendation. The method includes: obtaining at least one target user feature of a target user; for any one of the target user features, determining at least one recommended information with the highest relevance to any one of the target user features, and correcting the conversion rate of the target user clicking on the candidate recommended information according to the conversion rate deviation of the users with any one of the target user features who have clicked on the candidate recommended information, to obtain the corrected conversion rate of the target user clicking on the candidate recommended information; sorting the candidate recommended information according to the corrected conversion rate, and determining at least one recommended information to be recommended to the target user from the candidate recommended information according to the sorting result. The embodiment of the present application can effectively improve the estimation accuracy of the conversion rate, so as to more accurately recommend at least one recommended information to the target user.

Description

Technical Field

[0001] The present application relates to the technical field of information recommendation. Specifically, the present application relates to a method, apparatus, electronic device, and storage medium for determining recommended information. Background Art

[0002] The conversion rate is the proportion of Internet users who form conversions by clicking on recommended information to enter the promoted website. The calculation formula is the total number of conversions / the total number of clicks. Specific conversion behaviors can be defined by the advertiser independently. For example, behaviors such as registration and purchase can be used as conversion behaviors.

[0003] The conversion rate prediction model is an important part of the advertising recommendation system. The conversion rate prediction model is a model used to predict the conversion rate. After inputting the characteristics of the user and the advertisement characteristics, it can output the probability of the user generating a conversion behavior on the premise of clicking on the advertisement.

[0004] The prediction result of the conversion rate prediction model can directly affect parameters such as the conversion cost and the start-up rate of the advertisement. However, due to the strict time limit and computing power limit of model prediction, it is not possible to only consider the accuracy of the model and ignore the computing efficiency. In real advertising recommendation systems, a compromise solution is often adopted. On the premise that the performance consumption is satisfied, the complexity of the model is ensured as much as possible. Therefore, the complexity of the conversion rate prediction model often cannot reach the maximum, and the accuracy cannot reach the highest, thereby affecting the accurate placement of advertisements. Summary of the Invention

[0005] Embodiments of the present invention provide a method, apparatus, electronic device, and storage medium for determining recommended information that overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, a method for determining recommended information is provided. The method includes:

[0007] Obtain at least one target user characteristic of a target user;

[0008] For any one of the target user characteristics, determine at least one recommended information with the highest correlation with any one of the target user characteristics as the candidate recommended information corresponding to any one of the target user characteristics;

[0009] Obtain the recommended information characteristics of the candidate recommended information, and combine at least one target user characteristic to obtain the conversion rate after the target user clicks on the candidate recommended information;

[0010] Based on the conversion rate deviation of a candidate recommendation information clicked by a user with any one of the target user characteristics, correct the conversion rate after the target user clicks the candidate recommendation information to obtain the corrected conversion rate of the target user clicking the candidate recommendation information. The conversion rate deviation is obtained from the actual conversion rate and the predicted conversion rate after a sample user with any one of the target user characteristics clicks the candidate recommendation information;

[0011] Sort the candidate recommendation information according to the corrected conversion rate, and determine at least one recommendation information to be recommended to the target user from the candidate recommendation information according to the sorting result.

[0012] In a possible implementation, before determining at least one recommendation information with the highest correlation with any one of the target user characteristics as the candidate recommendation information, it further includes:

[0013] Obtain the historical behavior logs of at least one sample user, and construct an inverted index chain according to the historical behavior logs of at least one sample user. The historical behavior logs include the actual conversion behavior after the sample user clicks the recommendation information. The inverted index chain uses a user characteristic of the sample user determined from the historical behavior logs as the left key, and the nodes in the inverted index chain are used to represent the recommendation information with the highest correlation with the left key of the inverted index chain determined from the historical behavior logs;

[0014] Determining at least one recommendation information with the highest correlation with any one of the target user characteristics as the candidate recommendation information includes:

[0015] Search for the inverted index chain with any one of the target user characteristics as the left key as the target inverted index chain;

[0016] Use the recommendation information represented by the nodes on the target inverted index chain as the candidate recommendation information.

[0017] In a possible implementation, the historical behavior logs also include the predicted conversion rate after the sample user clicks the recommendation information;

[0018] Before correcting the conversion rate after the target user clicks the candidate recommendation information, it further includes:

[0019] According to any one of the sample user characteristics of the sample user, the predicted conversion rate and the actual conversion behavior after the sample user clicks the recommendation information, determine the conversion rate deviation of the sample user with any one of the sample user characteristics after clicking the recommendation information;

[0020] Determine the recommendation information represented by the nodes in the inverted index chain with any one of the sample user characteristics as the left key, and save the conversion rate deviation of the sample user with any one of the sample user characteristics after clicking the recommendation information represented by the nodes to the corresponding nodes;

[0021] Rectify the conversion rate after the target user clicks on the candidate recommendation information. Previously, it also included:

[0022] Obtain the conversion rate deviation after a user with target user characteristics clicks on the candidate recommendation information from the nodes on the target inverted index chain.

[0023] In a possible implementation, construct an inverted index chain based on the historical behavior logs of at least one sample user, including:

[0024] Mine the features of the historical behavior logs of at least one sample user to obtain the sample user characteristics of at least one sample user;

[0025] Input the sample user characteristics into a pre-trained recommendation model to obtain the information of at least one recommendation information with the highest correlation with the sample user characteristics output by the recommendation model;

[0026] Construct an inverted index chain with the sample user characteristics as the left key, and create the corresponding number of nodes in the inverted index chain according to the number of at least one recommendation information with the highest correlation with the sample user characteristics. Each node is used to represent one recommendation information with the highest correlation with the sample user characteristics.

[0027] In a possible implementation, determine the conversion rate deviation after a sample user with any sample user characteristic clicks on the recommendation information according to the estimated conversion rate and actual conversion behavior after the sample user clicks on the recommendation information, including:

[0028] Establish the corresponding relationship information among any sample user characteristic, the clicked recommendation information, the number of clicks, the number of conversions, and the estimated conversion rate of the sample user according to the historical behavior logs of the sample user and any sample user characteristic;

[0029] For any recommendation information involved in the historical behavior logs, determine the total number of clicks, the total number of conversions, and the average estimated conversion rate after all sample users with any sample user characteristic click on the recommendation information according to the corresponding relationship information;

[0030] Obtain the actual conversion rate after all sample users with any sample user characteristic click on the recommendation information according to the total number of clicks and the total number of conversions when all sample users with any sample user characteristic click on the recommendation information;

[0031] Obtain the conversion rate deviation after the sample user with the sample user characteristic clicks on the recommendation information according to the actual conversion rate and the average estimated conversion rate after all sample users with the sample user characteristic click on the recommendation information.

[0032] In a possible implementation, based on the actual conversion rate and the average predicted conversion rate after all sample users with any one sample user feature click on the recommended information, the conversion rate deviation after the sample users with any one sample user feature click on the recommended information is obtained, including:

[0033] Taking the quotient of the actual conversion rate and the average predicted conversion rate after all sample users with any one sample user feature click on the recommended information as the conversion rate deviation after the sample users with any one sample user feature click on the recommended information.

[0034] In a possible implementation, based on the conversion rate deviation of the user clicking on the candidate recommended information with any one target user feature that has been obtained, the conversion rate after the target user clicks on the candidate recommended information is corrected, including:

[0035] Obtaining the conversion rate deviation after the user with any one target user feature clicks on the candidate recommended information, which is related to all target user features of the target user, as the conversion rate deviation to be processed;

[0036] Determining the weighted average of the conversion rate deviation to be processed as the conversion rate deviation after the target user clicks on the candidate recommended information;

[0037] Correcting the conversion rate after the target user clicks on the candidate recommended information according to the conversion rate deviation after the target user clicks on the candidate recommended information.

[0038] In a possible implementation, based on the conversion rate deviation after the target user clicks on the candidate recommended information, the conversion rate after the target user clicks on the candidate recommended information is corrected, including:

[0039] Obtaining the corrected conversion rate of the target user clicking on the candidate recommended information according to the product of the conversion rate deviation after the target user clicks on the candidate recommended information and the conversion rate after the target user clicks on the candidate recommended information.

[0040] In a second aspect, a device for determining recommended information is provided, including:

[0041] A target user feature acquisition module, configured to acquire at least one target user feature of a target user;

[0042] A candidate recommended information acquisition module, configured to determine, for any one target user feature, at least one recommended information with the highest relevance to any one target user feature as the candidate recommended information corresponding to any one target user feature;

[0043] A conversion rate acquisition module, configured to acquire the recommended information feature of the candidate recommended information, and combine at least one target user feature to obtain the conversion rate after the target user clicks on the candidate recommended information;

[0044] A conversion rate correction module, which is used to correct the conversion rate after a target user clicks on candidate recommendation information according to the conversion rate deviation of a target user who clicks on candidate recommendation information and has any one of the target user characteristics, so as to obtain the corrected conversion rate of the target user clicking on the candidate recommendation information. The conversion rate deviation is obtained from the actual conversion rate and the estimated conversion rate after a sample user with any one of the target user characteristics clicks on the candidate recommendation information;

[0045] A recommendation module, which is used to sort the candidate recommendation information according to the corrected conversion rate, and determine at least one recommendation information to be recommended to the target user from the candidate recommendation information according to the sorting result.

[0046] In a possible implementation manner, the device for determining recommendation information further includes:

[0047] An inverted index chain construction module, which is used to obtain the historical behavior logs of at least one sample user, and construct an inverted index chain according to the historical behavior logs of at least one sample user. The historical behavior logs include the actual conversion behavior after the sample user clicks on the recommendation information. The inverted index chain uses a user characteristic of the sample user determined from the historical behavior logs as the left key, and the nodes in the inverted index chain are used to represent the recommendation information that has the highest correlation with the left key of the inverted index chain determined from the historical behavior logs;

[0048] The candidate recommendation information acquisition module includes:

[0049] A target inverted index chain acquisition sub-module, which is used to find an inverted index chain with any one of the target user characteristics as the left key as the target inverted index chain;

[0050] A node information acquisition module, which is used to use the recommendation information represented by the nodes on the target inverted index chain as the candidate recommendation information.

[0051] In a possible implementation manner, the historical behavior logs further include the estimated conversion rate after the sample user clicks on the recommendation information;

[0052] The device for determining recommendation information further includes:

[0053] A conversion rate deviation determination module, which is used to determine the conversion rate deviation of a sample user with any one of the sample user characteristics after clicking on the recommendation information according to any one of the sample user characteristics of the sample user, the estimated conversion rate and the actual conversion behavior after the sample user clicks on the recommendation information;

[0054] A conversion deviation storage module, which is used to determine the recommendation information represented by the nodes in the inverted index chain with any one of the sample user characteristics as the left key, and store the conversion rate deviation of the sample user with any one of the sample user characteristics after clicking on the recommendation information represented by the nodes into the corresponding nodes;

[0055] A conversion rate deviation acquisition module, configured to obtain the conversion rate deviation after a user with target user characteristics clicks on candidate recommendation information from the nodes on the target inverted list.

[0056] In a possible implementation manner, the inverted list construction module includes:

[0057] A feature mining sub-module, configured to perform feature mining on the historical behavior logs of at least one sample user to obtain the sample user characteristics of at least one sample user;

[0058] A recommendation information output sub-module, configured to input the sample user characteristics into a pre-trained recommendation model to obtain information about at least one recommendation information with the highest correlation with the sample user characteristics output by the recommendation model;

[0059] An inverted list construction sub-module, configured to construct an inverted list with the sample user characteristics as the left key, create corresponding numbers of nodes in the inverted list according to the number of at least one recommendation information with the highest correlation with the sample user characteristics, and each node is used to represent one recommendation information with the highest correlation with the sample user characteristics.

[0060] In a possible implementation manner, the conversion rate deviation determination module includes:

[0061] A correspondence determination sub-module, configured to establish correspondence information among any one sample user characteristic of the sample user, the recommended information clicked, the number of clicks, the number of conversions, and the estimated conversion rate according to the historical behavior logs of the sample user and any one sample user characteristic;

[0062] An intersection information acquisition sub-module, configured to, for any one recommended information involved in the historical behavior logs, determine the total number of clicks, the total number of conversions, and the average estimated conversion rate after all sample users with any one sample user characteristic click on the recommended information according to the correspondence information;

[0063] An actual conversion rate acquisition sub-module, configured to obtain the actual conversion rate after all sample users with any one sample user characteristic click on the recommended information according to the total number of clicks and the total number of conversions when all sample users with any one sample user characteristic click on the recommended information;

[0064] A conversion rate deviation acquisition sub-module, configured to obtain the conversion rate deviation after a sample user with sample user characteristics clicks on the recommended information according to the actual conversion rate and the average estimated conversion rate after all sample users with any one sample user characteristic click on the recommended information.

[0065] In a possible implementation, the conversion rate deviation acquisition sub-module is specifically configured to: use the quotient of the actual conversion rate and the average predicted conversion rate after all sample users with any one sample user feature click on the recommended information as the conversion rate deviation after the sample users with any one sample user feature click on the recommended information.

[0066] In a possible implementation, the conversion rate correction module includes:

[0067] A to-be-processed deviation acquisition sub-module, configured to obtain the conversion rate deviation after a user with any one target user feature clicks on the candidate recommended information, which is related to all target user features of the target user, as the to-be-processed conversion rate deviation;

[0068] A weighted average calculation sub-module, configured to determine the weighted average of the to-be-processed conversion rate deviation as the conversion rate deviation after the target user clicks on the candidate recommended information;

[0069] A correction sub-module, configured to correct the conversion rate after the target user clicks on the candidate recommended information according to the conversion rate deviation after the target user clicks on the candidate recommended information.

[0070] In a possible implementation, the correction sub-module is specifically configured to obtain the corrected conversion rate after the target user clicks on the candidate recommended information according to the product of the conversion rate deviation after the target user clicks on the candidate recommended information and the conversion rate after the target user clicks on the candidate recommended information.

[0071] In a third aspect, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method provided in the first aspect are implemented.

[0072] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method provided in the first aspect are implemented.

[0073] In a fifth aspect, an embodiment of the present invention provides a computer program, which includes computer instructions. The computer instructions are stored in a computer-readable storage medium. When a processor of a computer device reads the computer instructions from the computer-readable storage medium and the processor executes the computer instructions, the computer device is caused to execute the steps of the method provided in the first aspect.

[0074] The method, device, electronic device, and storage medium for determining recommended information provided by the embodiments of the present invention obtain the target user characteristics of the target user, determine the recommended information with the highest relevance to each target user characteristic as the candidate recommended information, and further obtain the conversion rate after the target user clicks on the candidate recommended information based on the recommended information characteristics of the candidate recommended information and in combination with the target user characteristics. By correcting the conversion rate after the target user clicks on the candidate recommended information using the conversion rate deviation of the user with any one of the target user characteristics who clicks on the candidate recommended information, the corrected conversion rate can be obtained. Since the conversion rate deviation takes into account the information in both the user characteristic and product dimensions, and associates both the actual conversion rate and the estimated conversion rate with the user characteristics and the product, it can effectively improve the estimation accuracy of the conversion rate compared to the defect of the dual tower model that ignores the cross-relationship between people and the recommended information, and thus more accurately recommend at least one piece of recommended information to the target user. Description of the Drawings

[0075] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for description in the embodiments of the present application.

[0076] Figure 1 It is a prediction logic diagram of an existing conversion rate prediction model;

[0077] Figure 2 Exemplarily shows the structural diagram of a system for determining recommended information provided by the embodiments of the present application;

[0078] Figure 3 Exemplarily shows the flowchart of the method for determining recommended information provided by the embodiments of the present application;

[0079] Figure 4 Exemplarily shows the structural diagram of an inverted index chain provided by the embodiments of the present application;

[0080] Figure 5 Exemplarily shows the flowchart of determining the conversion rate deviation after a sample user with any one of the sample user characteristics clicks on the recommended information provided by the embodiments of the present application;

[0081] Figure 6 Exemplarily shows the structural diagram of the inverted index chain for storing the conversion rate deviation provided by the embodiments of the present application;

[0082] Figure 7 Exemplarily shows the flowchart of correcting the conversion rate provided by the embodiments of the present application;

[0083] Figure 8 It is the structural diagram of a device for determining recommended information provided by the embodiments of the present application;

[0084] Figure 9A schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0085] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and should not be construed as a limitation of the present invention.

[0086] Those skilled in the art of the present technology can understand that unless specifically stated otherwise, the singular forms "a", "an" and "the" used herein may also include the plural forms. It should be further understood that the term "including" used in the specification of the present application means the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more related listed items.

[0087] To facilitate the understanding of the method provided by the embodiments of the present application, the nouns involved in the embodiments of the present application will be introduced first:

[0088] An advertising alliance generally refers to a network advertising alliance, which refers to a union formed by aggregating small and medium-sized network media resources (also known as alliance members, such as small and medium-sized websites, personal websites, WAP sites, etc.). Through the alliance, advertisers can achieve advertising placement and monitor and statistics the advertising placement data. Advertisers then pay advertising fees to alliance members according to the actual effects of network advertisements.

[0089] Advertisers: Those who want to recommend their own brands or products, such as BMW, Intel, Mengniu, Tencent, etc.

[0090] Recommendation information, that is, to inform the general public of something. Recommendation information can be either a non-profit announcement, statement, etc., or a commercial advertisement for profit. For commercial advertisements, the recommendation information contains information about the products to be recommended, and these information can further include elements such as categories, prices, names, etc.

[0091] Big data refers to a collection of data that cannot be captured, managed, and processed by conventional software tools within a certain time range. It is a vast, high-growth, and diverse information asset that requires new processing models to have stronger decision-making power, insight discovery ability, and process optimization ability. With the advent of the cloud era, big data has also attracted increasing attention. Big data requires special technologies to effectively process large amounts of data that can tolerate elapsed time. Technologies applicable to big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

[0092] Conversion Rate (CVR) is an indicator to measure the effect of CPA ads. In short, it is the conversion rate of users' conversion behavior after clicking on the ads. CVR = (Number of conversion behaviors / Number of clicks) * 100%.

[0093] pCVR: The probability of generating a conversion behavior under the premise of a click after exposing an ad to the audience is the predicted conversion rate.

[0094] eCPM (effective cost per mile): The advertising revenue that can be obtained for every thousand impressions.

[0095] pCTR (Predict Click-Through Rate): Predicted click-through rate.

[0096] oCPA (Optimized Cost Per Action): Optimized cost per action.

[0097] The conversion rate prediction model is used to predict the conversion rate based on user characteristics and product characteristics. In the rough ranking stage, since there are numerous ads to be ranked and the time requirement is more stringent, the widely adopted two-tower model in the industry is used for conversion rate prediction. However, this two-tower model has natural defects. It ignores the cross-relationship between people and ads, thus causing a certain loss to the prediction effect of the model.

[0098] Please refer to Figure 1 , Figure 1It is a prediction logic diagram of an existing conversion rate prediction model. During the rough ranking stage, the entire model is divided into two sides (each side will be referred to as a tower below). The tower on the left represents the sub-model related to users. The bottom layer of the tower is the user's characteristic information (such as gender, age, city, purchased goods, etc.). After passing through the embedding layer (vector layer), each piece of characteristic information becomes a vector; all the feature vectors are concatenated together, and then through multiple fully connected layers, all the user's vectors are transformed into a 32-dimensional vector. The right tower represents the information of the goods. The bottom layer is the characteristics of the goods (such as the merchant, category, name, brand of the goods, etc.). Similarly, the goods will also go through a series of transformations to become a 32-dimensional vector. The inner product of the user's vector and the goods' vector is calculated, and the obtained value can represent the predicted conversion rate after the user clicks on the goods advertisement. The larger the value, the higher the recommendation degree of the goods.

[0099] Although the existing conversion rate prediction model ensures the performance and time consumption of the model prediction, since the user's characteristics and the goods' characteristics are completely separated at the lowest end of the model and only converge together when calculating the inner product at the top layer, through analysis, it is found that this prediction method loses a large amount of information in the cross dimensions and is not conducive to ensuring the accuracy of the prediction.

[0100] The method, device, electronic device and storage medium for determining recommendation information provided by this application aim to solve the above technical problems of the existing technology.

[0101] To make the purpose, technical solution and advantages of this application clearer, the following will further describe the embodiments of this application in detail with reference to the accompanying drawings.

[0102] Figure 2 Exemplarily, a structural diagram of a system for determining recommendation information provided by an embodiment of this application is shown, as Figure 2 shown, the advertising alliance 11 is an alliance formed by integrating multiple traffic media platforms. The advertising alliance in the figure has three traffic media platforms. It should be noted that when actually applying the embodiments of this application, the number of traffic media platforms is not specifically limited. Here, it is only set to 3 traffic media platforms 111 for the convenience of narration. The background server of the traffic media platform 111 is the traffic media platform server 112. When the terminal 21 accesses the traffic media platform server 112, the traffic media platform server 112 provides traffic services. It can be understood that the traffic service can be to display a web page to the terminal. The traffic media platform server is connected to the advertising placement server 31 through the network. The advertising placement server 31 is used to execute the method for determining recommendation information in the following embodiments and send the finally determined recommendation information to the traffic media platform server 112, and the traffic media platform server 112 returns the web page with the recommendation information to the terminal.

[0103] Both the traffic media platform server and the advertisement placement server can be independent physical servers, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0104] The execution method of the server in the embodiments of the present application can be completed in the form of cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network that provides resources is called the "cloud". The resources in the "cloud" seem to be infinitely expandable to users, and can be obtained at any time, used on demand, expanded at any time, and paid according to usage.

[0105] As a basic capability provider of cloud computing, a cloud computing resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to select and use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0106] According to the logical function division, a PaaS (Platform as a Service) layer can be deployed on the IaaS (Infrastructure as a Service) layer, and a SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. Alternatively, the SaaS can be directly deployed on the IaaS. PaaS is a platform for software operation, such as databases, web containers, etc. SaaS is various business software, such as web portals, SMS mass senders, etc. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.

[0107] The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0108] Figure 3 Exemplarily, a flowchart of the method for determining recommended information in the embodiments of the present application is shown, as Figure 3 shown, including:

[0109] S101. Obtain at least one target user feature of the target user.

[0110] When a user is obtaining traffic services using a terminal, the traffic media platform server continuously collects the user's information and behaviors, then performs clustering analysis on this information to find the patterns therein, and uses these patterns to identify information such as the user's age, gender, favorite products, and the unique identifier of the terminal, etc., and uses them as the user's characteristics. It can be understood that the target user characteristics are the user characteristics identified for the target user. The embodiments of the present application do not make specific limitations on the number and type of target user characteristics.

[0111] S102. For any one target user characteristic, determine at least one recommended information with the highest correlation with any one target user characteristic as the candidate recommended information corresponding to any one target user characteristic.

[0112] In the current recommended information determination system, a recommendation model is usually trained using big data. By analyzing the user characteristics of each user in the big data and the recommended information highly relevant to each user, the recommended information corresponding to each type of user characteristic can be determined. Commonly, if the big data shows that users over 60 years old tend to be willing to buy health products, then for the user characteristic of being greater than 60 years old, the recommended information with high correlation can include health products. It should be understood that the correlation can represent the purchase intention, and the greater the correlation, the greater the purchase intention.

[0113] Therefore, for each target user characteristic, at least one recommended information with the highest correlation can be obtained. For example, if the target user has three target user characteristics, denoted as A1, A2, and A3 respectively, then at least one recommended information is determined for each of the 3 target user characteristics. For example, the recommended information with the highest correlation with A1 is B1 and B2, the recommended information with the highest correlation with A2 is B3 and B4, and the recommended information with the highest correlation with A3 is B5. Then 5 recommended information can be obtained for the target user. The embodiments of the present application do not limit the number of recommended information with high correlation for each target user characteristic.

[0114] S103. Obtain the recommended information characteristics of the candidate recommended information, and combine at least one target user characteristic to obtain the conversion rate after the target user clicks on the candidate recommended information.

[0115] It can be understood that similar to users, recommended information also has characteristics that characterize its own features, which are called recommended information characteristics. For a recommended information, the number of its recommended information characteristics is often greater than one. Taking the recommended information of a baby bottle as an example, its recommended information characteristics can include: maternal and infant products, materials, specific price ranges, etc.

[0116] After obtaining the recommended information characteristics of the candidate recommended information, it can be used Figure 1The shown two-tower model obtains the conversion rate after the target user clicks on the candidate recommendation information based on the target user characteristics of the target user and the recommendation information characteristics of the candidate recommendation information.

[0117] S104. According to the conversion rate deviation of the user with any one of the target user characteristics who clicks on the candidate recommendation information that has been obtained, correct the conversion rate after the target user clicks on the candidate recommendation information to obtain the corrected conversion rate after the target user clicks on the candidate recommendation information.

[0118] The conversion rate deviation in the embodiments of the present application is pre-obtained from the actual conversion rate and the estimated conversion rate after the sample user with any one of the target user characteristics clicks on the candidate recommendation information. Among them, the actual conversion rate after the sample user with any one of the target user characteristics clicks on the candidate recommendation information can be obtained by counting the number of clicks of the sample user with any one of the target user characteristics on the candidate recommendation information and the number of conversion behaviors sent after the sample user with any one of the target user characteristics clicks on the candidate recommendation information.

[0119] For example, if one of the target user characteristics of the target user is the "age group of 20-25 years old", and one of the candidate recommendation information is "iPhone", by collecting the historical behavior logs of a large number of sample users, screening out the historical behavior logs that simultaneously have the "age group of 20-25 years old" and "iPhone" for conversion behavior statistics. If the number of clicks of the sample users with the "age group of 20-25 years old" on the candidate recommendation information "iPhone" is 10,000, and the number of conversions of the sample users with the "age group of 20-25 years old" on the candidate recommendation information "iPhone" is 20, then the actual conversion rate of the sample users with the "age group of 20-25 years old" on the candidate recommendation information "iPhone" is 20 / 10000, which is 0.2%.

[0120] Similarly, when collecting the historical behavior logs of the sample users, the conversion rate after each sample user clicks on the recommendation information can be estimated. Furthermore, the average value of the estimated conversion rates of different sample users with the same sample user characteristics who click on the recommendation information can be taken to obtain the estimated conversion rate of the sample users with the same sample user characteristics who click on the recommendation information.

[0121] When calculating the conversion rate deviation in the present application, by simultaneously considering the information in two dimensions of user characteristics and commodities, both the actual conversion rate and the estimated conversion rate are simultaneously associated with user characteristics and commodities. Compared with the defect that the two-tower model ignores the cross-relationship between people and recommendation information, it can effectively improve the estimation accuracy of the conversion rate.

[0122] Since the conversion rate obtained in step S103 is an estimated conversion rate, according to the definition of the conversion rate deviation, it can be known that the corrected conversion rate is a more realistic conversion rate. When further determining the recommended information using the more realistic conversion rate, it can better meet the needs of the target user.

[0123] S105. Sort the candidate recommended information according to the corrected conversion rate, and determine at least one recommended information to be recommended to the target user from the candidate recommended information according to the sorting result.

[0124] The traffic price (i.e., eCPM) usually includes the CPM billing mode, the CPC billing mode, the CPA billing mode, and the oCPA billing mode. Among them, both the CPA billing mode and the oCPA billing mode are related to the conversion rate. Therefore, in the embodiments of the present application, after obtaining the corrected conversion rate, by substituting the corrected conversion rate and other parameters (such as the advertiser's bid, etc.) into the calculation formula of the billing mode, the sorting result of the revenue size of different candidate recommended information for the advertising alliance can be obtained. Based on this result, at least one candidate recommended information that meets the expected revenue for the advertising alliance can be obtained as the recommended information to be recommended to the target user.

[0125] The following takes the CPA billing mode as an example to illustrate the process of sorting the candidate recommended information after obtaining the corrected conversion rate in the embodiments of the present application.

[0126] The eCPM calculation formula for the CPA billing mode can be expressed as:

[0127] eCPM = bid × pCTR × pCVR

[0128] Wherein, bid represents the advertiser's bid, pCTR represents the estimated click-through rate, and pCVR represents the estimated conversion rate, which is the corrected conversion rate in the embodiments of the present application.

[0129] If the number of candidate recommended information is 3, which are candidate recommended information 1, candidate recommended information 2, and candidate recommended information 3 respectively. Specifically:

[0130] The bid of candidate recommended information 1 is 200 yuan per thousand impressions, the estimated click-through rate is 20%, and the corrected conversion rate is 4%;

[0131] The bid of candidate recommended information 2 is 240 yuan per thousand impressions, the estimated click-through rate is 18%, and the corrected conversion rate is 3.8%;

[0132] The bid of candidate recommended information 3 is 210 yuan per thousand impressions, the estimated click-through rate is 17%, and the corrected conversion rate is 4.2%;

[0133] By substituting the bid, pCTR, and pCVR of the three candidate recommendation information into the above calculation formula, the eCPM for candidate recommendation information 1 is 1.6, the eCPM for candidate recommendation information 2 is 1.64, and the eCPM for candidate recommendation information 3 is 1.49.

[0134] Sort the three candidate recommendation information in descending order according to their eCPM. The sorting result is candidate recommendation information 2, candidate recommendation information 1, and candidate recommendation information 3. If only one recommendation information is to be recommended to the target user, then recommendation information 2 will be recommended to the target user.

[0135] The method for determining recommendation information in the embodiments of the present application obtains the target user characteristics of the target user, determines the recommendation information with the highest relevance to each target user characteristic as the candidate recommendation information, further obtains the conversion rate after the target user clicks on the candidate recommendation information by combining the recommendation information characteristics of the candidate recommendation information with the target user characteristics, and corrects the conversion rate after the target user clicks on the candidate recommendation information by the conversion rate deviation of the user with any one target user characteristic who clicks on the candidate recommendation information. Thus, the corrected conversion rate can be obtained. Since the conversion rate deviation considers information in both the user characteristic and commodity dimensions, both the actual conversion rate and the estimated conversion rate are associated with the user characteristics and the commodity at the same time. Compared with the defect of the dual tower model that ignores the cross relationship between people and recommendation information, it can effectively improve the estimation accuracy of the conversion rate, and thus recommend at least one recommendation information to the target user more accurately.

[0136] Based on the above embodiments, when the present application determines the recommendation information with the highest relevance to the target user characteristics, it can be quickly obtained by using the inverted index chain method.

[0137] The inverted index chain is commonly used in the field of text analysis to record which documents contain a certain word. Generally, there are many documents in the document set that contain a certain word. Each document will record the document number (DocID), the number of times the word appears in the document (TF), and the positions where the word appears in the document, etc. The information related to a document is called an inverted index item (Posting). A series of inverted index items containing this word form a chain structure, which is the inverted index chain corresponding to this word. The left key of the inverted index chain is the word, and the nodes of the inverted index chain are the information of the word in the document. The embodiments of the present application draw on the existing concept of the inverted index chain, using the user characteristics as the left key and the recommendation information with the highest relevance to the user characteristics as the nodes.

[0138] Figure 4 Exemplarily shows a schematic structural diagram of an inverted index chain in the embodiments of the present application, as Figure 4As shown in the figure, the left key of the inverted index chain is the user feature "IT male", and there are 4 nodes in the inverted index chain, corresponding to the products "iPhone", "mechanical keyboard", "smart watch" and "shirt" respectively. By constructing an index system that stores the inverted index chain, during the search, only the target user feature "IT male" needs to be determined, and then the inverted index chain with the target user feature as the left key is queried in the index system, and the corresponding inverted index chain can be pulled out directionally, with very high efficiency.

[0139] Specifically, determining at least one recommended information with the highest relevance to any one target user feature as the candidate recommended information further includes:

[0140] Obtaining the historical behavior logs of at least one sample user, and constructing an inverted index chain according to the historical behavior logs of the at least one sample user.

[0141] As can be seen from the above embodiments, the historical behavior logs include the actual conversion behaviors after the sample users click on the recommended information. The actual conversion behaviors can be that the users do not have conversion behaviors, or the users generate conversion behaviors. It can be understood that the more abundant the historical behavior logs obtained in the embodiments of the present application involve products and sample users, the more beneficial it is to accurately obtain the recommended information.

[0142] By analyzing the historical behavior logs of the sample users, information such as the user features, product features, and the correlation between user features and different products of the sample users can be determined. Further, the above information can be used to establish an inverted index chain. The inverted index chain uses a user feature of a sample user determined from the historical behavior logs as the left key, and the nodes in the inverted index chain are used to represent the recommended information with the highest relevance to the left key of the inverted index chain determined from the historical behavior logs. The embodiments of the present application do not specifically limit the number of recommended information recorded in the inverted index chain. For example, it can be 5, 10, or even 20.

[0143] For example, if the products with the highest relevance to the users with the user feature of "mother" are "baby bottle", "diaper" and "toy" through analysis, an inverted index chain with the left key of "mother" and 3 nodes of "baby bottle", "diaper" and "toy" respectively can be generated.

[0144] Correspondingly, determining at least one recommended information with the highest relevance to any one target user feature as the candidate recommended information includes:

[0145] Searching for an inverted index chain with any one target user feature as the left key as the target inverted index chain;

[0146] Taking the recommended information represented by the nodes on the target inverted index chain as the candidate recommended information.

[0147] By constructing an inverted index chain with user features as the left key and the recommended information with the highest relevance to the user features as the nodes, considering that this application needs to correct the conversion rate of the target user clicking on the candidate recommended information, and the conversion rate of the target user clicking on the candidate recommended information is affected by each target user feature of the target user on the candidate recommended information. Therefore, the embodiments of this application can pre-set the conversion rate deviation of the target user feature on the candidate recommended information in the inverted index chain in advance. In this way, after obtaining the inverted index chain with the target user feature as the left key, the corresponding conversion rate deviation can be queried from the inverted index chain at the same time. Since the operation of querying the conversion rate deviation is a direct value-taking operation with a time complexity of 0(1), it hardly affects the time consumption and performance, which is equivalent to obtaining the candidate recommended information and the conversion rate deviation of the user with any one target user feature clicking on the candidate recommended information at the same time, greatly improving the determination efficiency of the recommended information.

[0148] Based on the above concept, the historical behavior log of the embodiments of this application further includes the estimated conversion rate after the sample user clicks on the recommended information.

[0149] Further, before correcting the conversion rate of the target user clicking on the candidate recommended information, it further includes:

[0150] S201. According to any one sample user feature of the sample user, the estimated conversion rate after the sample user clicks on the recommended information, and the actual conversion behavior, determine the conversion rate deviation of the sample user with any one sample user feature clicking on the recommended information.

[0151] Please refer to Figure 5 , which exemplarily shows the flow diagram of the embodiments of this application for determining the conversion rate deviation of the sample user with any one sample user feature clicking on the recommended information, as Figure 5 shown.

[0152] S2011. According to the historical behavior log of the sample user and any one sample user feature, establish the corresponding relationship information among any one sample user feature of the sample user, the recommended information clicked, the number of clicks, the number of conversions, and the estimated conversion rate.

[0153] It should be understood that a sample user with at least one sample user feature, after clicking on a product once, either has a conversion behavior or does not have a conversion behavior. Whether the conversion behavior occurs will be counted in the historical behavior log. At the same time, after the sample user clicks on the product, the estimated conversion rate will also be predicted according to the sample user feature of the sample user and the product feature of the product. That is to say, for the historical behavior log of one click behavior, multiple pieces of corresponding relationship information will be generated, and the difference between these multiple pieces of corresponding relationship information is only the different sample user features.

[0154] Specifically, a historical behavior log records the following information: After user Zhang San clicked on the recommended information for the iPhone, no conversion occurred, and at the same time, the estimated conversion rate for Zhang San to send a conversion behavior after clicking on the recommended information for the iPhone was 3%. Further, Zhang San's user characteristics include: aged 20 - 25, middle-income group, and otaku. Then, 3 pieces of corresponding relationship information can be obtained:

[0155] Corresponding relationship information 1: User characteristics: aged 20 - 25 — Recommended information: iPhone — Number of clicks: 1 — Number of conversions: 0 — Estimated conversion rate 3%

[0156] Corresponding relationship information 2: User characteristics: middle-income group — Recommended information: iPhone — Number of clicks: 1 — Number of conversions: 0 — Estimated conversion rate 3%

[0157] Corresponding relationship information 3: User characteristics: otaku — Recommended information: iPhone — Number of clicks: 1 — Number of conversions: 0 — Estimated conversion rate 3%.

[0158] S2012. For any recommended information involved in the historical behavior log, according to the corresponding relationship information, determine the total number of clicks, total number of conversions, and average value of the estimated conversion rate after all sample users with any one sample user characteristic click on the recommended information.

[0159] From the above example of the corresponding relationship information, by summarizing the corresponding relationship information with the same user characteristics and the same recommended information, the total number of clicks, total number of conversions, and average value of the estimated conversion rate after all sample users with the same user characteristics click on the same recommended information can be obtained. Among them, the average value of the estimated conversion rate is the average value of the estimated conversion rates included in the corresponding relationship information with the same user characteristics and the same recommended information.

[0160] For example, if it is statistically found that there are 3 pieces of information including the user characteristic "otaku" and the recommended information "Welding Technology Guide", which are respectively:

[0161] Corresponding relationship information a: User characteristics: otaku — Recommended information: Welding Technology Guide — Number of clicks: 1 — Number of conversions: 0 — Estimated conversion rate 1%

[0162] Corresponding relationship information b: User characteristics: otaku — Recommended information: Welding Technology Guide — Number of clicks: 1 — Number of conversions: 0 — Estimated conversion rate 0.7%

[0163] Corresponding relationship information c: User characteristics: otaku — Recommended information: Welding Technology Guide — Number of clicks: 1 — Number of conversions: 1 — Estimated conversion rate 0.1%.

[0164] Based on the above three pieces of corresponding relationship information, it can be determined that for all sample users with the user characteristic of "indoorsman", the total number of clicks after clicking on the recommended information "Welding Technology Guide" is 3, the total number of conversions is 1, and the average estimated conversion rate is 0.6% (the average of 1%, 0.7%, and 0.1%).

[0165] S2013. Obtain the actual conversion rate of all sample users with any one sample user characteristic after clicking on the recommended information based on the total number of clicks and the total number of conversions when all sample users with any one sample user characteristic click on the recommended information.

[0166] By dividing the total number of conversions by the total number of clicks, the actual conversion rate can be obtained. Continuing with the above corresponding relationship information a - c as an example, since the total number of clicks is 3 and the total number of conversions is 1, the actual conversion rate is 33%.

[0167] S2014. Obtain the conversion rate deviation of sample users with sample user characteristics after clicking on the recommended information based on the actual conversion rate of all sample users with any one sample user characteristic after clicking on the recommended information and the average estimated conversion rate.

[0168] Specifically, take the quotient of the actual conversion rate of all sample users with any one sample user characteristic after clicking on the recommended information and the average estimated conversion rate as the conversion rate deviation of sample users with any one sample user characteristic after clicking on the recommended information.

[0169] Continuing with the above corresponding relationship information a - c as an example, the conversion rate deviation of all sample users with the user characteristic of "indoorsman" after clicking on the recommended information "Welding Technology Guide" is 0.6% ÷ 33% = 0.018.

[0170] S202. Determine the recommended information represented by the nodes in the inverted index chain with any one sample user characteristic as the left key, and save the conversion rate deviation of the sample users with any one sample user characteristic after clicking on the recommended information represented by the nodes to the corresponding nodes.

[0171] In Figure 4 Based on the shown inverted index chain, if it is further determined that the conversion rate deviations of sample users with the user characteristic of "IT male" after clicking on the recommended information represented by "iPhone", "mechanical keyboard", "smart watch", and "shirt" are 1.2, 0.9, 1.1, and 1.0 respectively, then the inverted index chain after maintaining the conversion rate deviations in the corresponding nodes is as Figure 6 shown.

[0172] Based on the above embodiments, as an alternative embodiment, obtaining the conversion rate deviation of a user with any one target user characteristic after clicking on the candidate recommended information includes:

[0173] Obtain the conversion rate deviation after a user with target user characteristics clicks on the candidate recommendation information from the nodes on the target inverted index chain.

[0174] Take Figure 6 as an example. By querying the inverted index chain with the left key being "IT male", determine several recommendation information with the highest relevance to "IT male", namely "iPhone", "mechanical keyboard", "smart watch", and "shirt", according to the nodes of the inverted index chain. Then, obtain the conversion rate deviation from each of the 4 nodes respectively. The obtained conversion rate deviation is the conversion rate deviation after a user with the user characteristic of "IT male" clicks on the recommendation information "iPhone", "mechanical keyboard", "smart watch", and "shirt".

[0175] The product recall of the existing system is to establish the orientation of users (convert the user_tag into the id after hashing), and then pull out the inverted index chain of the product through the pre-established index system retrieved by the retrieval system through the orientation. Figure 7 Exemplarily shows the flow diagram of correcting the conversion rate in the embodiment of the present application, as Figure 7 shown:

[0176] By establishing the oriented search of users, pull out the inverted index chain with the user characteristic user_tag of the user as the left key. Through the inverted index chain, not only can several products (recommendation information) with the highest relevance to the user characteristic be obtained, but also the conversion rate deviation pcvr_bias after a user with the user characteristic clicks on the product (recommendation information) can be obtained.

[0177] Input the user characteristic of the user and the characteristics of the products in the inverted index chain into the conversion rate model to obtain the conversion rate output by the conversion rate model.

[0178] Finally, use the conversion rate deviation to correct the conversion rate, and the corrected conversion rate can be obtained.

[0179] Based on the above embodiments, as an optional embodiment, construct an inverted index chain according to the historical behavior logs of at least one sample user, including:

[0180] S301. Perform feature mining on the historical behavior logs of at least one sample user to obtain the sample user characteristics of at least one sample user;

[0181] When performing user feature mining, the historical behavior logs can also record more user-related information, such as the user's age, hobbies, registration time, login frequency, used items, cumulative consumption amount, etc. In addition, in LBS (Location Based Service) advertising, the distance between the user and the recommended information (advertisement) can also be used as one of the user characteristics to further correct the conversion rate prediction model.

[0182] S302. Input the sample user characteristics into a pre-trained recommendation model to obtain at least one recommendation information with the highest correlation with the sample user characteristics output by the recommendation model.

[0183] It should be understood that before step S302, the recommendation model can also be pre-trained. Specifically, the recommendation model can be trained in the following ways:

[0184] First, collect the historical recommendation logs of a certain number of sample users. The historical recommendation logs include the behavior records of users on the recommendation information. The behaviors can include positive behaviors, such as clicks, purchases, collections, etc., and can also include negative behaviors, such as rejecting recommendations, unfollowing, etc. By quantifying different behaviors, obtain the correlation results between the sample users and the recommendation information. Then, train the initial model based on the obtained user characteristics of the sample users and the correlation results between the sample users and the recommendation information. Among them, use the user characteristics of the sample users as the training samples, and use the correlation results between the sample users and the recommendation information as the sample labels to obtain the recommendation model.

[0185] Among them, the initial model can be a single neural network model or a combination of multiple neural network models.

[0186] S303. Construct an inverted index chain with the sample user characteristics as the left key, and create corresponding nodes in the inverted index chain according to the number of at least one recommendation information with the highest correlation with the sample user characteristics. Each node is used to represent one recommendation information with the highest correlation with the sample user characteristics.

[0187] Based on the above embodiments, as an optional embodiment, correct the conversion rate of the target user after clicking the candidate recommendation information according to the conversion rate deviation of the user with any one target user characteristic who clicks the candidate recommendation information obtained. The method includes:

[0188] S401. Obtain the conversion rate deviation of the user with any one target user characteristic who clicks the candidate recommendation information, which is related to all target user characteristics of the target user, as the conversion rate deviation to be processed.

[0189] Since the target user has at least one target user characteristic, and there is a conversion rate deviation between each target user characteristic and each candidate recommendation information, therefore, in the embodiments of the present application, it is necessary to obtain the conversion rate deviation of the user with any one target user characteristic who clicks the candidate recommendation information, which is related to all target user characteristics of the target user.

[0190] For example, if the target user has N target user characteristic information, which are T1, T2,..., T N, for the candidate recommendation information X1, the conversion rate deviation to be processed includes:

[0191] The conversion rate deviation P1 after a user with the target user feature T1 clicks on the candidate recommendation information X1;

[0192] The conversion rate deviation P2 after a user with the target user feature T2 clicks on the candidate recommendation information X1;

[0193] …

[0194] The conversion rate deviation P after a user with the target user feature T N clicks on the candidate recommendation information X1 N .

[0195] S402. Determine the weighted average of the conversion rate deviations to be processed as the conversion rate deviation of the target user clicking on the candidate recommendation information.

[0196] The weighted average is obtained by multiplying each conversion rate deviation to be processed by the corresponding weight, then summing up to get the total value, and then dividing by the total number of conversion rate deviations to be processed. In the embodiments of the present application, no specific limitation is imposed on the weights of the conversion rate deviations to be processed. A unified weight can be set, or different weights can be set according to the actual situation for different target user features. For example, if a user has two target user features, namely target user feature 1 and target user feature 2, and target user feature 1 has a greater impact on the user's acceptance of the recommendation information than target user feature 2, then the weight corresponding to the conversion rate deviation of target user feature 1 can be set higher than the weight corresponding to the conversion rate deviation of target user 2.

[0197] S403. Correct the conversion rate of the target user clicking on the candidate recommendation information according to the conversion rate deviation of the target user clicking on the candidate recommendation information.

[0198] Specifically, the corrected conversion rate of the target user clicking on the candidate recommendation information is obtained according to the product of the conversion rate deviation of the target user clicking on the candidate recommendation information and the conversion rate of the target user clicking on the candidate recommendation information.

[0199] The embodiments of the present application provide a device for determining recommendation information. As Figure 8 shown, the device may include: a target user feature acquisition module 101, a candidate recommendation information acquisition module 102, a conversion rate acquisition module 103, a conversion rate correction module 104, and a recommendation module 105. Specifically:

[0200] The target user feature acquisition module 101 is used to acquire at least one target user feature of the target user;

[0201] The candidate recommendation information acquisition module 102 is configured to, for any target user feature, determine at least one recommendation information with the highest relevance to any target user feature as the candidate recommendation information corresponding to any target user feature;

[0202] The conversion rate acquisition module 103 is configured to acquire the recommendation information features of the candidate recommendation information, and combine at least one target user feature to obtain the conversion rate after the target user clicks on the candidate recommendation information;

[0203] The conversion rate correction module 104 is configured to correct the conversion rate after the target user clicks on the candidate recommendation information according to the conversion rate deviation of the user with any target user feature who clicks on the candidate recommendation information, so as to obtain the corrected conversion rate after the target user clicks on the candidate recommendation information. The conversion rate deviation is obtained from the actual conversion rate and the predicted conversion rate after the sample user with any target user feature clicks on the candidate recommendation information;

[0204] The recommendation module 105 is configured to sort the candidate recommendation information according to the corrected conversion rate, and determine at least one recommendation information to be recommended to the target user from the candidate recommendation information according to the sorting result.

[0205] The recommendation information determination device provided by the embodiment of the present invention specifically executes the process of the above method embodiment. For details, please refer to the content of the above recommendation information determination method embodiment, which will not be elaborated here. The recommendation information determination device provided by the embodiment of the present invention determines the recommendation information with the highest relevance to each target user feature as the candidate recommendation information by acquiring the target user features of the target user. Further, according to the recommendation information features of the candidate recommendation information, the conversion rate after the target user clicks on the candidate recommendation information is obtained by combining the target user features. The conversion rate after the target user clicks on the candidate recommendation information is corrected by the conversion rate deviation of the user with any target user feature who clicks on the candidate recommendation information, and the corrected conversion rate can be obtained. Since the conversion rate deviation takes into account the information in both the user feature and commodity dimensions, both the actual conversion rate and the predicted conversion rate are associated with the user features and commodities at the same time. Compared with the defect of the dual tower model that ignores the cross relationship between people and recommendation information, it can effectively improve the prediction accuracy of the conversion rate, so as to more accurately recommend at least one recommendation information to the target user.

[0206] In a possible implementation manner, the recommendation information determination device further includes:

[0207] An inverted index building module, which is used to obtain the historical behavior logs of at least one sample user, build an inverted index according to the historical behavior logs of at least one sample user, where the historical behavior logs include the actual conversion behaviors of the sample users after clicking the recommended information. The inverted index uses a user feature of a sample user determined from the historical behavior logs as the left key, and the nodes in the inverted index are used to represent the recommended information that has the highest correlation with the left key of the inverted index and is determined from the historical behavior logs;

[0208] The candidate recommended information acquisition module includes:

[0209] A target inverted index acquisition sub-module, which is used to find an inverted index with any target user feature as the left key as the target inverted index;

[0210] A node information acquisition module, which is used to use the recommended information represented by the nodes on the target inverted index as the candidate recommended information.

[0211] In a possible implementation, the historical behavior logs also include the estimated conversion rate of the sample users after clicking the recommended information;

[0212] The determining device for the recommended information further includes:

[0213] A conversion rate deviation determination module, which is used to determine the conversion rate deviation of the sample users with any sample user feature after clicking the recommended information according to any sample user feature of the sample users, the estimated conversion rate of the sample users after clicking the recommended information, and the actual conversion behavior;

[0214] A conversion deviation storage module, which is used to determine the recommended information represented by the nodes in the inverted index with any sample user feature as the left key, and store the conversion rate deviation of the sample users with any sample user feature after clicking the recommended information represented by the nodes into the corresponding nodes;

[0215] A conversion rate deviation acquisition module, which is used to obtain the conversion rate deviation of the users with the target user feature after clicking the candidate recommended information from the nodes on the target inverted index.

[0216] In a possible implementation, the inverted index building module includes:

[0217] A feature mining sub-module, which is used to perform feature mining on the historical behavior logs of at least one sample user to obtain the sample user features of at least one sample user;

[0218] A recommended information output sub-module, which is used to input the sample user features into a pre-trained recommendation model to obtain the information of at least one recommended information with the highest correlation with the sample user features output by the recommendation model;

[0219] An inverted index building sub-module, which is used to build an inverted index with sample user features as the left key, and create corresponding nodes in the inverted index according to the number of at least one recommended information with the highest relevance to the sample user features. Each node is used to represent one recommended information with the highest relevance to the sample user features.

[0220] In a possible implementation, the conversion rate deviation determination module includes:

[0221] A correspondence determination sub-module, which is used to establish the correspondence information among any sample user feature of the sample user, the recommended information clicked, the number of clicks, the number of conversions, and the estimated conversion rate according to the historical behavior log of the sample user and any sample user feature;

[0222] An intersection information acquisition sub-module, which is used to determine the total number of clicks, the total number of conversions, and the average value of the estimated conversion rate after all sample users with any sample user feature click the recommended information according to the correspondence information for any recommended information involved in the historical behavior log;

[0223] An actual conversion rate acquisition sub-module, which is used to obtain the actual conversion rate after all sample users with any sample user feature click the recommended information according to the total number of clicks and the total number of conversions when all sample users with any sample user feature click the recommended information;

[0224] A conversion rate deviation acquisition sub-module, which is used to obtain the conversion rate deviation after the sample users with sample user features click the recommended information according to the actual conversion rate and the average value of the estimated conversion rate after all sample users with any sample user feature click the recommended information.

[0225] In a possible implementation, the conversion rate deviation acquisition sub-module is specifically used to: take the quotient of the actual conversion rate and the average value of the estimated conversion rate after all sample users with any sample user feature click the recommended information as the conversion rate deviation after the sample users with any sample user feature click the recommended information.

[0226] In a possible implementation, the conversion rate correction module includes:

[0227] A to-be-processed deviation acquisition sub-module, which is used to obtain the conversion rate deviation after the users with any target user feature related to all target user features of the target user click the candidate recommended information as the to-be-processed conversion rate deviation;

[0228] A weighted average value calculation sub-module, which is used to determine the weighted average value of the to-be-processed conversion rate deviation as the conversion rate deviation after the target user clicks the candidate recommended information;

[0229] A deviation correction sub-module, configured to correct the conversion rate after the target user clicks on the candidate recommendation information according to the conversion rate deviation after the target user clicks on the candidate recommendation information.

[0230] In a possible implementation manner, the deviation correction sub-module is specifically configured to obtain the corrected conversion rate after the target user clicks on the candidate recommendation information according to the product of the conversion rate deviation after the target user clicks on the candidate recommendation information and the conversion rate after the target user clicks on the candidate recommendation information.

[0231] In the embodiments of the present application, an electronic device is provided. The electronic device includes: a memory and a processor; at least one program, stored in the memory and configured to, when executed by the processor, compared with the prior art, achieve: by obtaining the target user characteristics of the target user, determining the recommendation information with the highest correlation with each target user characteristic as the candidate recommendation information, further obtaining the conversion rate after the target user clicks on the candidate recommendation information according to the recommendation information characteristics of the candidate recommendation information and combining the target user characteristics, and correcting the conversion rate after the target user clicks on the candidate recommendation information through the conversion rate deviation of the user with any one of the target user characteristics who has clicked on the candidate recommendation information that has been obtained, the corrected conversion rate can be obtained. Since the conversion rate deviation takes into account the information in both the user characteristics and product dimensions, both the actual conversion rate and the estimated conversion rate are associated with the user characteristics and the product at the same time. Compared with the defect that the dual tower model ignores the cross relationship between people and the recommendation information, it can effectively improve the estimation accuracy of the conversion rate, so as to more accurately recommend at least one piece of recommendation information to the target user.

[0232] In an alternative embodiment, an electronic device is provided, as Figure 9 shown. Figure 9 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. Among them, the processor 4001 and the memory 4003 are connected, such as connected through a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation to the embodiments of the present application.

[0233] The processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of this application. The processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0234] The bus 4002 may include a path for transmitting information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 only a thick line is shown herein, but it does not mean that there is only one bus or one type of bus.

[0235] The memory 4003 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or it may also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0236] The memory 4003 is used to store the application program code for executing the solution of this application, and is controlled by the processor 4001 for execution. The processor 4001 is used to execute the application program code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0237] An embodiment of this application provides a computer-readable storage medium, on which a computer program is stored. When it runs on a computer, it enables the computer to execute the corresponding content in the foregoing method embodiments. Compared with the prior art, by obtaining the target user characteristics of the target user, determining the recommendation information with the highest correlation with each target user characteristic as the candidate recommendation information, further obtaining the conversion rate after the target user clicks the candidate recommendation information according to the recommendation information characteristics of the candidate recommendation information and combining the target user characteristics, and correcting the conversion rate after the target user clicks the candidate recommendation information through the conversion rate deviation of the user with any one target user characteristic who has clicked the candidate recommendation information, the corrected conversion rate can be obtained. Since the conversion rate deviation takes into account the information in both the user characteristic and product dimensions, both the actual conversion rate and the estimated conversion rate are associated with the user characteristics and products at the same time. Compared with the defect of the twin tower model that ignores the cross relationship between people and recommendation information, it can effectively improve the estimation accuracy of the conversion rate, so as to more accurately recommend at least one piece of recommendation information to the target user.

[0238] An embodiment of this application provides a computer program, which includes computer instructions. The computer instructions are stored in a computer-readable storage medium. When the processor of the computer device reads the computer instructions from the computer-readable storage medium and the processor executes the computer instructions, the computer device executes the content shown in the foregoing method embodiments. Compared with the prior art, by obtaining the target user characteristics of the target user, determining the recommendation information with the highest correlation with each target user characteristic as the candidate recommendation information, further obtaining the conversion rate after the target user clicks the candidate recommendation information according to the recommendation information characteristics of the candidate recommendation information and combining the target user characteristics, and correcting the conversion rate after the target user clicks the candidate recommendation information through the conversion rate deviation of the user with any one target user characteristic who has clicked the candidate recommendation information, the corrected conversion rate can be obtained. Since the conversion rate deviation takes into account the information in both the user characteristic and product dimensions, both the actual conversion rate and the estimated conversion rate are associated with the user characteristics and products at the same time. Compared with the defect of the twin tower model that ignores the cross relationship between people and recommendation information, it can effectively improve the estimation accuracy of the conversion rate, so as to more accurately recommend at least one piece of recommendation information to the target user.

[0239] It should be understood that although the steps in the flowchart of the accompanying drawings are shown sequentially according to the indication of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear indication in this document, there is no strict order restriction for the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0240] The above are only some embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for determining recommended information, characterized in that, Including: Obtaining at least one target user feature of a target user; For any one of the target user features, determining at least one recommended information with the highest correlation with the any one of the target user features as candidate recommended information corresponding to the any one of the target user features; Obtaining recommended information features of the candidate recommended information, and combining the at least one target user feature to obtain a conversion rate after the target user clicks the candidate recommended information; According to the conversion rate deviation of a user with the any one of the target user features who clicks the candidate recommended information that has been obtained, correcting the conversion rate after the target user clicks the candidate recommended information to obtain a corrected conversion rate after the target user clicks the candidate recommended information, where the conversion rate deviation is obtained from the actual conversion rate and the estimated conversion rate after a sample user with the any one of the target user features clicks the candidate recommended information; Sorting the candidate recommended information according to the corrected conversion rate, and determining at least one recommended information to be recommended to the target user from the candidate recommended information according to the sorting result; Among them, before determining at least one recommended information with the highest correlation with the any one of the target user features as candidate recommended information, it further includes: Obtaining historical behavior logs of at least one sample user, and constructing an inverted index chain according to the historical behavior logs of the at least one sample user. The historical behavior logs include the actual conversion behavior after the sample user clicks the recommended information. The inverted index chain uses a user feature of the sample user determined from the historical behavior logs as the left key, and the nodes in the inverted index chain are used to represent the recommended information with the highest correlation with the left key of the inverted index chain determined from the historical behavior logs; Determining at least one recommended information with the highest correlation with the any one of the target user features as candidate recommended information includes: Searching for an inverted index chain with the any one of the target user features as the left key as the target inverted index chain; Taking the recommended information represented by the nodes on the target inverted index chain as the candidate recommended information.

2. The method for determining recommended information according to claim 1, wherein The historical behavior logs further include the estimated conversion rate after the sample user clicks the recommended information; Before correcting the conversion rate after the target user clicks the candidate recommended information, it further includes: According to any one of the sample user features of the sample user, the estimated conversion rate and the actual conversion behavior after the sample user clicks the recommended information, determining the conversion rate deviation after the sample user with the any one of the sample user features clicks the recommended information; Determining the recommended information represented by the nodes in the inverted index chain with the any one of the sample user features as the left key, and saving the conversion rate deviation after the sample user with the any one of the sample user features clicks the recommended information represented by the nodes to the corresponding nodes; Before correcting the conversion rate after the target user clicks the candidate recommended information, it further includes: Obtaining the conversion rate deviation after the user with the target user feature clicks the candidate recommended information from the nodes on the target inverted index chain.

3. The method for determining recommended information according to claim 1, wherein Constructing an inverted index based on the historical behavior logs of at least one sample user includes: Performing feature mining on the historical behavior logs of the at least one sample user to obtain the sample user features of the at least one sample user; Inputting the sample user features into a pre-trained recommendation model to obtain information on at least one recommended information with the highest correlation with the sample user features output by the recommendation model; Constructing an inverted index with the sample user features as the left key, creating a corresponding number of nodes in the inverted index according to the number of at least one recommended information with the highest correlation with the sample user features, and each node is used to represent one recommended information with the highest correlation with the sample user features.

4. The method for determining recommended information according to claim 2, wherein Determining the conversion rate deviation after a sample user with any one of the sample user features clicks on a recommended information according to any one of the sample user features of the sample user, the estimated conversion rate after the sample user clicks on the recommended information, and the actual conversion behavior, includes: Establishing corresponding relationship information among any one of the sample user features of the sample user, the recommended information clicked, the number of clicks, the number of conversions, and the estimated conversion rate according to the historical behavior logs of the sample user and any one of the sample user features; For any one of the recommended information involved in the historical behavior logs, determining the total number of clicks, the total number of conversions, and the average estimated conversion rate after all sample users with any one of the sample user features click on the recommended information according to the corresponding relationship information; Obtaining the actual conversion rate after all sample users with any one of the sample user features click on the recommended information according to the total number of clicks and the total number of conversions when all sample users with any one of the sample user features click on the recommended information; Obtaining the conversion rate deviation after a sample user with the sample user features clicks on the recommended information according to the actual conversion rate and the average estimated conversion rate after all sample users with any one of the sample user features click on the recommended information.

5. The method for determining recommended information according to claim 4, wherein Obtaining the conversion rate deviation after a sample user with any one of the sample user features clicks on the recommended information according to the actual conversion rate and the average estimated conversion rate after all sample users with any one of the sample user features click on the recommended information, includes: Taking the quotient of the actual conversion rate and the average estimated conversion rate after all sample users with any one of the sample user features click on the recommended information as the conversion rate deviation after a sample user with any one of the sample user features clicks on the recommended information.

6. The method for determining recommended information according to claim 2, wherein Correcting the conversion rate after the target user clicks on the candidate recommended information according to the conversion rate deviation of the user with any one of the target user features who has clicked on the candidate recommended information that has been obtained, includes: Obtaining the conversion rate deviation after the user with any one of the target user features related to all the target user features of the target user clicks on the candidate recommended information as the conversion rate deviation to be processed; Determine the weighted average of the to-be-processed conversion rate deviation as the conversion rate deviation after the target user clicks on the candidate recommendation information; Correct the conversion rate after the target user clicks on the candidate recommendation information according to the conversion rate deviation after the target user clicks on the candidate recommendation information.

7. The method for determining recommended information according to claim 6, wherein The correcting the conversion rate after the target user clicks on the candidate recommendation information according to the conversion rate deviation after the target user clicks on the candidate recommendation information includes: Obtain the corrected conversion rate after the target user clicks on the candidate recommendation information according to the product of the conversion rate deviation after the target user clicks on the candidate recommendation information and the conversion rate after the target user clicks on the candidate recommendation information.

8. An apparatus for determining recommendation information, characterized in that including: A target user feature acquisition module for acquiring at least one target user feature of a target user; A candidate recommendation information acquisition module for, for any one target user feature, determining at least one recommendation information with the highest relevance to the any one target user feature as the candidate recommendation information corresponding to the any one target user feature; A conversion rate acquisition module for acquiring the recommendation information features of the candidate recommendation information and combining the at least one target user feature to obtain the conversion rate after the target user clicks on the candidate recommendation information; A conversion rate correction module for correcting the conversion rate after the target user clicks on the candidate recommendation information according to the conversion rate deviation of the user with the any one target user feature who has clicked on the candidate recommendation information that has been obtained, and obtaining the corrected conversion rate after the target user clicks on the candidate recommendation information, where the conversion rate deviation is obtained from the actual conversion rate and the estimated conversion rate after the sample user with the any one target user feature clicks on the candidate recommendation information; A recommendation module for sorting the candidate recommendation information according to the corrected conversion rate and determining at least one recommendation information to be recommended to the target user from the candidate recommendation information according to the sorting result; Among them, before the candidate recommendation information acquisition module determines at least one recommendation information with the highest relevance to the any one target user feature as the candidate recommendation information, it is also used for: Obtain the historical behavior logs of at least one sample user, and construct an inverted index chain according to the historical behavior logs of at least one sample user. The historical behavior logs include the actual conversion behavior after the sample user clicks on the recommendation information. The inverted index chain uses a user feature of the sample user determined from the historical behavior logs as the left key, and the nodes in the inverted index chain are used to represent the recommendation information with the highest relevance to the left key of the inverted index chain determined from the historical behavior logs; The candidate recommendation information determining at least one recommendation information with the highest relevance to the any one target user feature as the candidate recommendation information includes: Search for the inverted index chain with the left key being the any one target user feature as the target inverted index chain; Use the recommendation information represented by the nodes on the target inverted index chain as the candidate recommendation information.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for determining the recommended information as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause the computer to execute the steps of the method for determining the recommended information as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes computer instructions that are stored in a computer-readable storage medium. When the processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, the computer device is caused to execute the steps of the method for determining the recommended information as described in any one of claims 1 - 7.

Images