A model training method, an advertisement material prediction method and device

Abstract

The application provides a model training method, an advertisement material prediction method and device. The model training method comprises: determining a training sample set based on an advertisement material set with call end records; determining advertisement material basic features, advertisement material playing features, user basic features, user historical behavior features, and cross features between the advertisement material and the user corresponding to the training sample; inputting the advertisement material basic features, the advertisement material playing features, the user basic features, the user historical behavior features, and the cross features between the advertisement material and the user corresponding to the training sample into an initial advertisement material scoring model for prediction processing to obtain first prediction scores of the advertisement material corresponding to the training sample; and adjusting parameters of the advertisement material scoring model according to the first prediction scores of the advertisement material corresponding to the training sample and positive and negative sample labels of the training sample until a convergence condition is reached to obtain a trained advertisement material scoring model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model training method, an advertising material prediction method, and an apparatus. Background Technology

[0002] As a social platform with a massive user base, Weibo offers various advertising types, including splash screen ads and feed ads. The advertising platform provides rich audience targeting features, allowing for precise targeting based on users' age, gender, location, interests, and other dimensions.

[0003] However, the current sampling process is negative sampling at the call end user level. The sampling ratio of each user is fixed, and channels and specifications cannot be distinguished, which will generate a lot of invalid samples, thus making it impossible to accurately deliver high-quality advertising materials. Summary of the Invention

[0004] This application provides a model training method, an advertising creative prediction method, and an apparatus to achieve accurate delivery of high-quality advertising creatives.

[0005] The technical solution of this application is implemented as follows:

[0006] This application provides a training method for a model, the method comprising: determining a training sample set based on a set of advertising creatives with caller records; caller records are used to record the caller behavior of users clicking on advertising creatives exposed on an advertising platform to enter the advertiser's designated page; the training sample set includes multiple training samples corresponding to each advertising creative, wherein positive samples in the multiple training samples are matching pairs between advertising creatives and corresponding clicking users, and negative samples in the multiple training samples are matching pairs between advertising creatives and non-clicking users, where clicking users refer to users who have caller behavior for advertising creatives, and non-clicking users refer to users who have not had caller behavior for advertising creatives but have caller behavior for other advertising creatives in the same advertising space; determining the advertising creative base corresponding to the training samples. The initial ad creative scoring model is formed by inputting the following features: basic features, ad creative playback features, user basic features, user historical behavior features, and cross-features between ad creatives and users. This input includes the basic features of the ad creatives, ad creative playback features, user basic features, user historical behavior features, and cross-features between ad creatives and users corresponding to the training samples. The model then performs prediction processing to obtain the first predicted score for the ad creatives in the training samples. This first predicted score represents the degree of user interest in the ad creatives in the training samples. Based on the first predicted score of the ad creatives in the training samples and the positive and negative sample labels of the training samples, the parameters of the ad creative scoring model are adjusted until convergence is achieved, resulting in the trained ad creative scoring model.

[0007] This application provides a method for predicting advertising creatives. The method includes: acquiring the basic features and playback features of the target advertising creative to be predicted, the basic user features and historical user behavior features of multiple active users corresponding to the ad slot to which the target advertising creative belongs, and the cross features between the target advertising creative and each active user; wherein, the multiple active users corresponding to the ad slot refer to the set of users who have call-to-the-end behavior for any advertising creative in the ad slot within a preset time period; inputting the basic features and playback features of the target advertising creative, the basic user features and historical user behavior features of multiple active users, and the cross features into a trained advertising creative scoring model for prediction processing to obtain multiple second predicted scores corresponding to the target advertising creative; the advertising creative scoring model is trained using the training method of the above model, and the second predicted scores are used to characterize the degree of interest of active users in the target advertising creative; based on the multiple second predicted scores corresponding to the target advertising creative, the total predicted score of the target advertising creative corresponding to multiple active users is determined, and the target advertising creatives are ranked based on the total predicted score.

[0008] This application provides a training device for a model, comprising: a sampling module, used to determine a training sample set based on a set of advertising creatives with caller records; the caller records are used to record the caller behavior of users clicking on advertising creatives exposed on an advertising platform to enter the advertiser's designated page; the training sample set includes multiple training samples corresponding to each advertising creative, wherein positive samples among the multiple training samples are matching pairs between advertising creatives and corresponding clicking users, and negative samples among the multiple training samples are matching pairs between advertising creatives and non-clicking users, where clicking users refer to users who have caller behavior for advertising creatives, and non-clicking users refer to users who have not had caller behavior for advertising creatives but have caller behavior for other advertising creatives in the same advertising space; and a data processing module, used to determine the advertising creative base corresponding to the training samples. The system includes basic features, ad creative playback features, user basic features, user historical behavior features, and cross-features between ad creatives and users; a scoring training module, which inputs the basic features of ad creatives, ad creative playback features, user basic features, user historical behavior features, and cross-features between ad creatives and users corresponding to the training samples into the initial ad creative scoring model for prediction processing, to obtain the first predicted score corresponding to the ad creative in the training samples; the first predicted score is used to characterize the degree of interest of users in the training samples in the ad creatives; a training module, which adjusts the parameters of the ad creative scoring model according to the first predicted score corresponding to the ad creative in the training samples and the positive and negative sample labels of the training samples, until the convergence condition is reached, to obtain the trained ad creative scoring model.

[0009] This application provides an advertising creative prediction device, comprising: a data acquisition module for acquiring the basic features and playback features of the target advertising creative to be predicted, the basic user features and historical user behavior features of multiple active users corresponding to the ad slot to which the target advertising creative belongs, and the cross features between the target advertising creative and each active user; wherein, the multiple active users corresponding to the ad slot refer to the set of users who have call-to-the-end behavior for any advertising creative in the ad slot within a preset time period; a scoring module for inputting the basic features and playback features of the target advertising creative, the basic user features and historical user behavior features of multiple active users, and the cross features into a trained advertising creative scoring model for prediction processing to obtain multiple second predicted scores corresponding to the target advertising creative; the advertising creative scoring model is trained using the above model training method, and the second predicted scores are used to characterize the degree of interest of active users in the target advertising creative; and a ranking module for determining the total predicted score of the target advertising creative corresponding to multiple active users based on the multiple second predicted scores, so as to rank the target advertising creative based on the total predicted score.

[0010] This application provides an electronic device, including: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the method provided in this application.

[0011] This application provides a computer storage medium storing executable instructions, which are executed by a processor to implement the method provided in this application.

[0012] This application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the method provided in this application.

[0013] This application has the following beneficial effects:

[0014] In this application, positive and negative samples are sampled based on the exposure and call logs of the advertising platform and a preset overall sampling ratio. This enables dynamic sampling at the creative granularity level based on the exposure and click data of the advertising creative, optimizes the negative sample sampling method, and improves the quality of both positive and negative samples. The advertising creative scoring model trained based on this can achieve accurate targeting of high-quality creatives.

[0015] Furthermore, adding ad placements as a sampling dimension during the sampling process can more closely reflect programmatic customer acquisition scenarios; and from the perspective of material granularity, a higher-quality database can be built.

[0016] Furthermore, in conjunction with actual programmatic customer acquisition business scenarios, playback features that can effectively evaluate video quality have been added. Utilizing existing caller data, a matching score for each set of ad creatives is calculated by combining interest tags, tag scores, and duration coefficients, thus constructing user historical behavior features that reflect user interests and preferences. The introduction of cross-features increases the overall feature dimensionality of the ad creative scoring model and improves its ability to fit complex data. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the architecture of the recommendation system provided in the embodiments of this application;

[0018] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0019] Figure 3 This is an optional flowchart illustrating the training method of the model provided in the embodiments of this application;

[0020] Figure 4 This is an optional flowchart illustrating the training method of the model provided in the embodiments of this application;

[0021] Figure 5 This is a schematic diagram of the advertising material scoring model provided in the embodiments of this application;

[0022] Figure 6 This is an optional flowchart illustrating the advertising material prediction method provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0025] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0027] Weibo's external advertising channels are primarily managed manually, resulting in low customer acquisition efficiency. To improve customer acquisition efficiency across these channels, a large-scale, high-quality creative library can be built, and machine learning-based advertising models can be introduced across multiple channels to score and rank ad creatives, thereby enhancing the efficiency and effectiveness of user acquisition and engagement. Compared to conventional manual buying, this approach can significantly improve the efficiency, scale, and delivery strategy of ad purchasing.

[0028] Programmatic user acquisition is a technique used for off-platform advertising on Weibo. It involves identifying high-quality materials within the Weibo context, generating creative ad creatives, and then placing and managing these ads through a demand-side platform (DSP). In Weibo's programmatic user acquisition, the ad delivery model is cost-per-click (CPC), meaning the advertiser's bid per click multiplied by the estimated click-through rate (pCTR). The higher the product of the bid-per-click and the estimated CTR, the higher the probability that the ad creative will be exposed to users through competition. Therefore, with the same bid-per-click, there are two main ways to improve the platform's estimated pCTR: 1. Upgrading ad creatives: designing excellent ad creatives to present the content to users in a more effective way; 2. Optimizing programmatic user acquisition ad creatives: selecting truly high-quality creatives from the massive daily ad creative pool using ranking algorithms to provide to the ad delivery platform for higher exposure opportunities. Upgrading ad creatives is often limited by the actual ad delivery platform; therefore, optimizing ad creatives from the advertiser's perspective is the fundamental way to increase ad exposure. Current programmatic customer acquisition ranking systems use random negative sampling at the user level to construct a training dataset and train a model, combining known basic features of creative materials and user characteristics. Then, daily selected advertising creative materials are matched with designated user packages, and the model makes predictions and scores them. Each advertising creative material is voted on and scored by setting a threshold. The top N advertising creative materials are selected as high-quality creative materials based on the scoring results.

[0029] However, in programmatic customer acquisition scenarios, advertisers cannot obtain real creative exposure data, and the difficulty in data collection leads to a lack of negative samples for model training. Furthermore, negative sampling at the user level has the disadvantage that the sampling ratio for each user is fixed, and it is impossible to distinguish between channels and specifications, resulting in a large number of invalid samples.

[0030] Furthermore, currently, 99% of programmatic advertising placements utilize video creatives. Besides the basic characteristics of video creatives, video playback data can also serve as an important indicator of ad creative quality. However, the limited scope and data available in a single programmatic scenario make it impossible to obtain reasonable video playback data. In the creative scoring stage, the method of using a threshold-based voting system—scoring 1 for exceeding the threshold and 0 for otherwise—can differentiate between different creatives. However, the threshold setting requirements are very high, and the differentiation is insufficient. This can easily lead to situations where creatives of relatively average quality receive the same score, resulting in an inability to accurately target high-quality ad creatives.

[0031] To address the aforementioned issues, this application provides a model training method to achieve precise targeting of high-quality advertising materials.

[0032] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as various types of user terminals such as laptops, tablets, desktop computers, and mobile devices (e.g., mobile phones, wearable smartwatches, dedicated messaging devices), or as servers. The following will describe exemplary applications when the electronic device is implemented as a server.

[0033] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the recommendation system provided in the embodiments of this application. Electronic devices (electronic devices 40-1 and 40-2 are shown as examples) are connected to server 200 through network 300. Network 300 can be a wide area network or a local area network, or a combination of both.

[0034] In some possible implementations, user A can publish material information A through terminal 40-1, and user B can publish material information B through terminal 40-2. Material information A and material information B are uploaded to server 200 via network 300. Server 200 can store material information A and material information B in database 500. To achieve precise material recommendation based on user interests, server 200 can sort the materials and push them to the user with the highest degree of matching with the user's interests. The materials pushed to the user can be displayed on the graphical interface of electronic device 400 (graphical interfaces 41-1 and 41-2 are shown as examples).

[0035] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The terminal may be a smartphone, tablet, laptop, desktop computer, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment of the invention.

[0036] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 2 The illustrated electronic device 400 can be the aforementioned terminal and / or server 200. Electronic device 400 includes: at least one processor 410, memory 450, at least one network interface 420, and user interface 430. The various components in electronic device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0037] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0038] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0039] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0040] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0041] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0042] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0043] The network communication module 452 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0044] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0045] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0046] In some embodiments, the training device for the model and the advertising material prediction device provided in this application can be implemented in software. Figure 2 The training device 455 and the creative material prediction device 456 of the model stored in the memory 450 are shown. The model training device 455 can be software in the form of programs and plug-ins. The training device 455 includes the following software modules: sampling module 4551, data processing module 4552, scoring training module 4553 and training module 4554. The creative material prediction device 456 includes the following software modules: data acquisition module 4561, scoring module 4562 and sorting module 4563.

[0047] These modules are logically structured, and therefore can be combined or further broken down arbitrarily according to their implemented functions. The functions of each module will be explained below.

[0048] In other embodiments, the training device and advertising material prediction device of the model provided in this application embodiment can be implemented in hardware. As an example, the device provided in this application embodiment can be a processor in the form of a hardware decoding processor, which is programmed to execute the training method and advertising material prediction method of the model provided in this application embodiment. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0049] In some embodiments, the recommendation system provided in this application can be a material recommendation system. The training method of the model provided in this application will be described below using a material recommendation system as an example.

[0050] The training method of the model provided in the embodiments of this application will be described below with reference to exemplary applications and implementations of the electronic devices provided in the embodiments of this application.

[0051] It should be noted that, for ease of explanation, the training device for the model can be simply referred to as the training device.

[0052] See Figure 3 , Figure 3 This is a schematic diagram of an optional training method for the model provided in the embodiments of this application. The following will be combined with... Figure 3 The steps shown illustrate the training method for the model.

[0053] It should be noted that the advertising creatives are off-site advertisements targeting the material recommendation system, placed on other applications on the user's device. The advertising creatives can be materials published within the material recommendation system itself. That is, advertising creatives are generated based on materials published by the material recommendation system and placed in appropriate ad slots. Users can click on the advertising creative in the ad slot to enter the in-app page of the application corresponding to the material recommendation system. The application corresponding to the material recommendation system can be Weibo.

[0054] S301, Based on the set of advertising materials with caller records, determine the training sample set.

[0055] In some embodiments, call recording is used to record the call behavior of users clicking on an advertiser's designated page through an ad creative exposed on the ad delivery platform; the training sample set includes multiple training samples corresponding to each ad creative, wherein the positive samples in the multiple training samples are matching pairs between the ad creative and the corresponding clicking users, and the negative samples in the multiple training samples are matching pairs between the ad creative and non-clicking users. Clicking users refer to users who have call behavior for the ad creative, and non-clicking users refer to users who have no call behavior for the ad creative but have call behavior for other ad creatives in the same ad slot.

[0056] In some embodiments, the ad delivery platform's exposure logs record the exposure of multiple ad creatives corresponding to each ad placement. For example, ad creative a in ad placement A was exposed 100 times in the past 24 hours, and ad creative b in ad placement A was exposed 150 times in the past 24 hours. Call logs record user call behavior (i.e., click behavior) of the ad creatives; for example, ad creative a in ad placement A was clicked 50 times in the past 24 hours, and ad creative b in ad placement A was clicked 65 times in the past 24 hours.

[0057] In some embodiments, ad creatives that are exposed and clicked by users can serve as positive samples, such as (s i (u1) indicates that user u1 clicked on the ad material s i , (s i u1) is advertising material s i A corresponding positive sample. Ad creatives that are displayed but not clicked by users can be used as negative samples, for example (s). i u m+1 ) represents user u m+1 Unclicked ad creatives i , (s i u m+1 ) for advertising materials i This corresponds to a negative sample. It should be noted that when the same ad creative is exposed multiple times, it may be clicked by some users and not by others. Therefore, the same ad creative can appear as both a positive and a negative sample.

[0058] In some possible implementations, S301 may include: determining, based on the call logs of the advertising platform, a set of advertising creatives with call records within a preset first time period, and a positive sample corresponding to each advertising creative in the set of advertising creatives; wherein, the call logs include all call records in the advertising platform, and the number of positive samples corresponding to the advertising creatives is equal to the number of users who have call behavior for the advertising creatives; determining, based on the exposure logs, call logs, and a preset overall sampling ratio of the advertising platform, a negative sample corresponding to each advertising creative in the set of advertising creatives; wherein, the exposure logs include exposure records of each advertising creative in the advertising platform, the overall sampling ratio is the ratio between positive and negative samples set for the ad placement, and the number of negative samples corresponding to the advertising creatives is determined based on the exposure volume, call volume, and overall sampling ratio corresponding to the advertising creatives.

[0059] Understandably, the number of users clicking on each ad creative can be collected from the call logs of the ad delivery platform. Based on the number of users clicking on each ad creative and the number of users who clicked on the ad creative, a positive sample set corresponding to each ad creative can be generated. The number of positive samples in the positive sample set is the number of users who clicked on each ad creative. To avoid the generation of invalid negative samples, in this embodiment of the application, the negative samples are artificially constructed. It is necessary to first determine the number of negative samples corresponding to each ad creative and then determine the negative sample set.

[0060] In some embodiments, the collection date of the ad creative set is preset, such as determining the ad creative set with caller records based on the caller log updated the previous day.

[0061] In some embodiments, call logs are used to record user click behavior on sample ad creatives; the overall sampling ratio is the ratio between positive and negative samples of all sample ad creatives in the ad slot.

[0062] In one example, advertising creatives i The corresponding set of positive samples can be represented as {(s i ,u1),(s i ,u2),…,(s i u m That is, user u1, user u2, ..., user u m For each advertising material s i A click occurred.

[0063] In one example, advertising creatives i The corresponding set of negative samples can be represented as That is, user u m+1 User u m+2 ,…,user No ad creativesi A click occurred.

[0064] In some possible implementations, determining the negative samples corresponding to each ad creative in the ad creative set based on the exposure logs, call logs, and a preset overall sampling ratio of the ad delivery platform may include: for any ad creative in the ad creative set, obtaining the exposure volume of the ad creative and the total exposure volume of all ad creatives in the ad slot to which the ad creative belongs from the exposure logs; obtaining the call volume of the ad creative and the total call volume of all ad creatives in the ad slot to which the ad creative belongs based on the call logs; the call volume is used to characterize the number of users who have call behavior for the ad creative; calculating the negative sample coefficient corresponding to the ad creative based on the overall sampling ratio, the total exposure volume of all ad creatives, and the total call volume of all ad creatives; multiplying the difference between the exposure volume and the call volume of the ad creative by the negative sample coefficient to obtain the number of negative samples corresponding to the ad creative; and selecting a corresponding number of non-clicking users of the ad creative from the negative sample candidate set of the ad slot to which the ad creative belongs, based on the number of negative samples corresponding to the ad creative, to form the negative samples corresponding to the ad creative; wherein, the negative sample candidate set is the set of users who have call behavior for any ad creative in the same ad slot.

[0065] Understandably, the sum of the impressions of each ad creative within the same ad slot equals the total impressions of all ad creatives. Similarly, the sum of the call counts of each ad creative equals the total call count of all ad creatives. The overall sampling ratio is preset and is the ratio of the number of positive samples to the number of negative samples.

[0066] In some embodiments, the expression (1) for the negative sample coefficient is as follows:

[0067]

[0068] In equation (1), r is the negative sample coefficient, R is the overall sampling ratio, C is the total number of calls to all advertising materials, and E is the total number of exposures to all advertising materials. According to equation (1), the negative sample coefficients of each advertising material in the same advertising slot are the same.

[0069] In some embodiments, the expression (2) for calculating the number of negative samples corresponding to each ad creative is as follows:

[0070] N m =r×(e i -c i (2)

[0071] In equation (2), r is the negative sample coefficient, and N m For advertising materials iThe number of negative samples, e i For advertising materials i Exposure, c i For advertising materials i The volume of exhaled air.

[0072] In some embodiments, for the same advertising creative, the positive-to-negative sample ratio can be expressed as m∶r×(e i -c i The ratio of positive to negative samples and the e-value of advertising materials. i -c i It is inversely proportional, that is, e i -c i The larger the positive-to-negative sample ratio, the smaller the e i -c i The smaller the value, the larger the ratio of positive to negative samples. And e i -c i This reflects the exposure effectiveness of the ad creative; a smaller difference indicates a higher percentage of clicks and better performance. High-performing ad creatives have higher sample importance and greater information value. Increasing the proportion of important samples will benefit the training of the model (i.e., the ad creative scoring model).

[0073] In some embodiments, after determining the number of negative samples corresponding to the ad creative, a corresponding number of non-clicking users can be selected from the non-clicking users of the ad creative. The matching pairs between these non-clicking users and the ad creative constitute the negative sample set.

[0074] In some embodiments, advertising creatives i Exposure to m+r i ×(e i -c i ) users, the total user set is represented as Here, the user set corresponding to the positive sample is denoted as {u1,u2,…,um}. After filtering out the user set corresponding to the positive sample from the total user set, r is randomly selected from the remaining users. i ×(e i -c i ) users are taken as the user set corresponding to the negative sample. The user set corresponding to the negative sample can be represented as:

[0075] In some embodiments, after determining the number of negative samples in the negative sample set corresponding to each advertising material according to the above calculation method, the above sampling method can be verified. The expression (3) of the derivation process of the verification formula is as follows:

[0076]

[0077] In equation (3), the advertising materials for the same advertising space are represented as {s1,s2,…,s}. n}, e i For advertising materials i Exposure, c i For advertising materials i The call volume. Equation (3) verifies that the above sampling scheme can achieve e-based sampling while ensuring the overall sampling ratio is controllable. i -c i Dynamic sampling.

[0078] In this embodiment, the negative sampling method, which randomly selects ad creatives that have not been clicked from the user's perspective, is optimized to a sampling method that dynamically samples ad creatives based on their exposure and click performance, targeting users who have not clicked. This changes the random sampling scope from all ad creatives to active users of the corresponding ad placement, increasing ad placement differentiation and aligning more closely with programmatic customer acquisition scenarios. Furthermore, from the perspective of ad creatives, it better reflects the algorithm design intent of building a high-quality creative library. Additionally, the sampling method changes from a uniform proportion for each user to dynamic sampling based on the exposure and click performance of the creatives. This increases the sampling quantity for high-performing and more valuable ad creatives and decreases the sampling quantity for low-performing ones, improving the overall quality of both positive and negative samples while keeping the total number of samples constant.

[0079] S302, determine the basic features of the advertising creative, the playback features of the advertising creative, the basic features of the user, the historical behavior features of the user, and the cross features between the advertising creative and the user corresponding to the training sample.

[0080] Understandably, based on the database of the advertising platform and the positive and negative sample sets corresponding to each advertising creative, it is possible to determine the user's basic characteristics and historical behavior characteristics, the advertising creative's basic characteristics and playback characteristics, and the cross-features between each user and each advertising creative.

[0081] In some embodiments, the training device can pair multiple users and multiple advertising creatives one-to-one to obtain the cross-features between each user and each advertising creative. For example, users include user u1 and user u2, and advertising creatives include s1 and s2; users and advertising creatives can form 4 matching pairs, namely (u1, s1), (u1, s2), (u2, s1), and (u2, s2).

[0082] In some possible implementations, S302 above may include: the training device acquiring basic user features of users in the training samples from the user feature library of the material recommendation system; acquiring basic advertising material features of advertising materials in the training samples from the material library of the material recommendation system; acquiring user history behavior features of users in the training samples from the call logs of the advertising platform; acquiring advertising material playback features of advertising materials in the training samples from the full-site video playback database of the material recommendation system; and determining the cross features between users and advertising materials in the training samples based on user information and material information of advertising materials.

[0083] In some embodiments, the user feature library of the material recommendation system stores the basic user features of users. These basic user features are used to characterize a user's personal profile; for example, they may represent information such as the user's age, frequency of logging into the material recommendation system, primary interest tag, city of residence, mobile phone brand, and model, and may also include other dimensions of information, which are not specifically limited in this embodiment.

[0084] In some embodiments, the material library of the material recommendation system stores the basic characteristics of each advertising material. The basic characteristics of the advertising material are used to characterize the material information of the advertising material. For example, the material characteristics can represent information such as material tags, second interest tags, the advertising position to which the material belongs, the number of images, the video duration, the static characteristics of the material blogger, and the number of days the material is exposed. It can also include information of other dimensions. This application embodiment does not specifically limit this.

[0085] In some possible implementations, obtaining the user history behavior characteristics of users in the training sample from the call logs of the advertising platform may include: obtaining multiple historical advertising materials clicked by users in the training sample within a preset second time period from the call logs; determining the duration coefficient of multiple historical advertising materials based on the call logs; the duration coefficient is used to represent the difference between the month in which the historical advertising material was published and the current month; obtaining the interest tags and tag scores corresponding to each historical advertising material; dividing the multiple historical advertising materials into multiple groups of historical advertising materials according to the interest tags; each group of historical advertising materials corresponds to one interest tag; calculating the user's matching score for each group of historical advertising materials based on the tag score and duration coefficient corresponding to each historical advertising material; sorting the multiple matching scores, and using the multiple groups of historical advertising materials whose matching scores meet a preset threshold as the user's user history behavior characteristics.

[0086] Understandably, a sliding window approach can be used to statistically analyze the call data in the call logs to obtain the historical behavioral characteristics of each user. The size of the window (i.e., the second time period) can be preset, such as one year.

[0087] In some embodiments, the training device obtains the release time of historical ad creatives from the call logs, subtracts the current month from the month of the release time, and the difference is the duration coefficient of the historical ad creative. For example, if it is currently December, and the call time of the i-th historical ad creative occurred in October, then the duration coefficient t of the i-th historical ad creative is... i =2.

[0088] In some embodiments, each historical ad creative has a corresponding interest tag (i.e., a second interest tag), which indicates the interest area to which the ad creative belongs. The information related to the interest tags of historical ad creatives is stored in the database of the ad creative recommendation system.

[0089] In one example, the second interest tag for historical ad creative A is "entertainment stars", the second interest tag for historical ad creative B is "variety shows", and the second interest tag for historical ad creative C is "animals".

[0090] In some embodiments, each historical ad creative corresponds to a tag score, which is stored in the database of the ad creative recommendation system. While determining the second interest tag corresponding to each historical ad creative, the tag score corresponding to each historical ad creative can be obtained.

[0091] In some embodiments, after determining the second interest tag for each historical ad creative, historical ad creatives belonging to the same second interest tag can be grouped together. That is, multiple historical ad creatives are grouped into multiple sets of historical ad creatives, each set including at least one historical ad creative, and each set of historical ad creatives corresponding to one interest tag.

[0092] In one example, historical ad creatives A and D both have the secondary interest tag "entertainment stars," historical ad creatives B and E both have the secondary interest tag "variety shows," and historical ad creatives C and F both have the secondary interest tag "animals." Therefore, historical ad creatives A and D can be grouped together, and this group of historical ad creatives belongs to the same category of interest tags; historical ad creatives B and E can be grouped together, and this group of historical ad creatives belongs to the same category of interest tags; historical ad creatives C and F can be grouped together, and this group of historical ad creatives belongs to the same category of interest tags.

[0093] In some embodiments, the expression (4) for calculating the user's matching score for each set of historical ad creatives based on the tag score and duration coefficient corresponding to each historical ad creative is as follows:

[0094]

[0095] In equation (4), the number of historical advertising materials is n, and S a t represents the match score of this group of historical ad creatives with interest tag 'a'. i f represents the duration coefficient of the i-th historical ad creative. i This represents the tag score of the i-th historical ad creative for interest tag a.

[0096] In one example, taking the interest tags, tag scores and duration coefficients shown in Table 1 as examples, the matching scores corresponding to the three groups of historical advertising materials are calculated according to expression (4).

[0097] Table 1

[0098]

[0099] In some embodiments, taking Table 1 as an example, after calculating the matching scores of the three sets of historical ad creatives, these three sets of historical ad creatives can be sorted from largest to smallest. The historical ad creatives corresponding to the top two ranked ones are selected as the user's historical behavior characteristics; then, referring to Table 1, the user's historical behavior characteristics are ad creatives a and b belonging to "entertainment stars"; and ad creatives c, d, and e belonging to "variety shows".

[0100] In some possible implementations, playback features may include one or any combination of the following: actual plays, actual play rate, average play rate, x% completion rate, and completion rate; wherein, actual plays represent the number of times the ad creative is played; actual play rate represents the ratio between the number of times the ad creative is played and the number of times the ad creative is exposed; average play rate represents the average of the actual play rates of all ad creatives in the same ad slot; x% completion rate represents the ratio between the number of plays and actual plays when the ratio of the duration of the user's play of the ad creative to the total duration of the ad creative is greater than x%; and completion rate represents the ratio between the number of times the user fully plays the ad creative and the actual plays.

[0101] In some possible implementations, obtaining the playback characteristics corresponding to the advertising materials in the training samples from the site's full-site video playback database of the material recommendation system may include: the training device obtaining the actual playback volume and exposure volume corresponding to the advertising materials in the training samples from the site's full-site video playback database; based on the actual playback volume and exposure volume corresponding to the advertising materials; calculating the actual playback rate and average playback rate; obtaining the playback duration corresponding to the advertising materials from the site's full-site video playback database; when the advertising materials are played completely, obtaining the completion rate of the advertising materials by calculating the ratio of the number of times the advertising materials are played completely to their corresponding actual playback volume; when the advertising materials are not played completely, obtaining the x% completion rate of the advertising materials by calculating the ratio of the number of times the playback duration percentage of the advertising materials is greater than x% to their corresponding actual playback volume.

[0102] Understandably, in programmatic customer acquisition scenarios, 99% of ad creatives are video creatives. Therefore, this application embodiment introduces playback characteristics that better reflect the quality of ad creatives. Specifically, the x% completion rate can include: 10% completion rate, 30% completion rate, 50% completion rate, and 80% completion rate.

[0103] In some embodiments, video play counts are recorded from the moment the video begins playing. However, this statistical logic is susceptible to user errors such as accidental clicks, taps, or instant exits, leading to statistical results that are often higher than the actual video play count. The actual play count described in this application's embodiments is based on a set threshold, such as only starting to count play counts when the playback duration exceeds 5% of the total video duration. This statistical method more closely reflects actual video playback conditions.

[0104] In some embodiments, the true play rate is the ratio between the actual number of times an ad creative is played and the number of times it is exposed; the average play rate is the average of the true play rates of all ad creatives in the same ad slot.

[0105] Understandably, besides statistical characteristics like play count, play rate can also significantly reflect the quality of ad creatives. Ad creatives with high play rates, even with low play counts, are often high-quality ad creatives.

[0106] In some embodiments, the site-wide video playback database stores data tables that record video playback data for the entire site. In programmatic scenarios, where the timeliness of advertising creatives is crucial, a sliding window approach can be used to retrieve playback data from the previous week for statistical analysis.

[0107] In some embodiments, the playback duration corresponding to each ad creative is obtained from the site-wide video playback database; if the ad creative is played in its entirety (i.e., the ratio of the playback duration of the ad creative to the total duration of the ad creative is 1), the completion rate of the ad creative is calculated.

[0108] In some embodiments, when an ad creative is not fully played, the x% completion rate of each ad creative is obtained by calculating the ratio of the number of times the playback duration of each ad creative is greater than x% to its corresponding actual playback volume.

[0109] In one example, taking a 50% completion rate as an example, calculate the ad creative s i The expression for a 50% completion rate (5) is as follows:

[0110]

[0111] In equation (5), W 50％ Indicates advertising material s i A 50% completion rate, br represents the ad creative's s i The actual playback duration, t represents the video duration of the ad creative si, (b r / t)>50% means that the ratio of the actual playback time of the advertising material si to the video length is greater than 50%. (b) r / t)>5% means that the ratio of the actual playback time of the advertising material si to the video length is greater than 5%, and Num() represents the quantity.

[0112] It should be noted that the calculation method for other x% completion rates is the same as the calculation method for the above-mentioned 50% completion rate, and this application embodiment does not specifically limit this.

[0113] In some possible implementations, the cross-features may include typological cross-features. The method of determining the cross-features between users and advertising materials in the training samples based on user information and advertising material material information may include: determining a first tag corresponding to the user and a second tag corresponding to the advertising material based on the user information and advertising material material information in the training samples; if the first tag and the second tag are the same, determining the typological cross-feature between the user corresponding to the first tag and the advertising material corresponding to the second tag as a first typological cross-feature; if the first tag and the second tag are different, determining the typological cross-feature between the user corresponding to the first tag and the advertising material corresponding to the second tag as a second typological cross-feature.

[0114] Understandably, based on a user's information, multiple primary tags can be identified for that user. For example, if user A's province is "Shaanxi" and their city is "Xi'an," then user A's primary tags could include "Shaanxi" and "Xi'an." Similarly, based on the material information of an advertising creative, multiple secondary tags can be identified for that creative. For instance, if the blogger who posted advertising creative A's province is "Sichuan" and their city is "Chengdu," then advertising creative A's secondary tags could include "Sichuan" and "Chengdu."

[0115] In some embodiments, regarding province information, user A's first tag is "Shaanxi" and advertising material A's second tag is "Sichuan". Since user A's first tag and advertising material A's second tag are the same, the cross feature of province information between user A and advertising material A is determined as a first type of cross feature, and the first type of cross feature can be assigned a value of 1.

[0116] In some embodiments, regarding province information, user A's first tag is "Shaanxi" and advertising material B's second tag is "Shaanxi". Since user A's first tag and advertising material B's second tag are the same, the cross feature of province information between user A and advertising material B is determined as a second type of cross feature. The second type of cross feature can be assigned a value of 0.

[0117] In some possible implementations, the cross-features may further include a first numerical cross-feature and a second numerical cross-feature; the first tag includes multiple first interest tags, and the second interest tag includes multiple second interest tags. The above-mentioned determination of the cross-features between users and advertising materials in the training samples based on user information and advertising material material information in the training samples may include: determining multiple first interest tags corresponding to users in the training samples, and determining multiple second interest tags corresponding to advertising materials in the training samples; performing matching processing on the multiple first interest tags of users and the multiple second interest tags of advertising materials in the training samples, and determining the number of mutually corresponding first interest tags and second interest tags as the first numerical cross-features between users and advertising materials; obtaining preset interest tag weights when there are first numerical cross-features between users and advertising materials; one first interest tag corresponds to one interest tag weight; and determining the second numerical cross-features between users and advertising materials based on the first numerical cross-features and interest tag weights.

[0118] Understandably, primary interest tags are used to characterize a user's areas of interest. For example, user A's areas of interest are "entertainment stars" and "variety shows," which are user A's two primary interest tags. Secondary interest tags are used to characterize the areas of interest of advertising creatives. For example, advertising creative A's areas of interest are "finance" and "funds," which are advertising creative A's two secondary interest tags.

[0119] In some embodiments, when the user and the ad creative belong to the same area of ​​interest, the first interest tag and the second interest tag correspond to each other (i.e., the first interest tag and the second interest tag are the same). For example, if user B and ad creative C both have the area of ​​interest "pets", then user B's first interest tag and ad creative C's second interest tag correspond to each other.

[0120] In some embodiments, a user's interest domain and an ad creative's interest domain can share multiple common domains. For example, user C's interest domains include "variety shows," "entertainment stars," and "TV dramas." Ad creative D's interest domains include "entertainment stars" and "TV dramas." Therefore, user C and ad creative D have two corresponding interest tags: "entertainment stars" and "TV dramas," and the first numerical cross-feature between user C and ad creative D is 2.

[0121] In some embodiments, each user has multiple first interest tags, each with a corresponding interest tag weight, which can be obtained from the database of the material recommendation system.

[0122] In some embodiments, when the first interest tag and the second interest tag match (i.e. are the same), the average matching score between the first interest tag and the second interest tag (i.e. the value of the second numerical cross feature) can represent the user's level of interest in the advertising material.

[0123] In some embodiments, the expression (6) for determining the second numerical cross feature between the user and the advertising creative, based on the first numerical cross feature and the interest tag weight, is as follows:

[0124]

[0125] In equation (6), Avg score Indicates user u j With advertising materials i The average matching score between users, u j For each of the n primary interest tags, w represents user u. j The corresponding interest tag weight of the first interest tag, user u j The weights correspond to the n interest tags.

[0126] In some embodiments, the cross features include 3 types of cross types and 2 numerical cross types, which are summarized in Table 2.

[0127] Table 2

[0128] Feature Name Types of cross features Did the user and the content blogger's province match? Category cross features Did the user and the content blogger's city match? Category cross features Do the first interest tag and the second interest tag match? Category cross features Hit count of first interest tag and second interest tag First numerical cross feature Mean matching score between the first interest tag and the second interest tag Second numerical crossover feature

[0129] In Table 2, "material blogger" is a shorthand for bloggers who publish advertising materials.

[0130] S303, the training device inputs the basic features of the advertising creative, the playback features of the advertising creative, the basic features of the user, the historical behavior features of the user, and the cross-features between the advertising creative and the user corresponding to the training sample into the initial advertising creative scoring model for prediction processing, and obtains the first predicted score corresponding to the advertising creative in the training sample. The first predicted score is used to characterize the degree of interest of users in the training sample towards the advertising creative.

[0131] Understandably, after obtaining the basic user characteristics and historical behavior characteristics for each user, and the basic ad creative characteristics, ad creative playback characteristics, and cross-features between the ad creative and the user for each ad creative, these characteristics are input into the ad creative scoring model for prediction and scoring. The ad creative scoring model can output a first predicted score for each ad creative. The higher the first predicted score, the higher the user's interest in the ad creative.

[0132] In some embodiments, during the training of the advertising creative scoring model, user basic features, user historical behavior features, advertising creative basic features, advertising creative playback features, and cross features between advertising creative and users can be input into the advertising creative scoring model in the form of vectors for training.

[0133] In some embodiments, Figure 4 This is a schematic diagram of an optional training method for the model provided in this application embodiment. See also... Figure 4 As shown, the advertising material scoring model is illustrated using the FM (factorization machine) model as an example.

[0134] S401, the training device collects positive and negative samples based on exposure logs and call logs.

[0135] S402, the training device generates user basic features, user historical behavior features, advertising material basic features, advertising material playback features and cross features based on positive sample sets and negative sample sets.

[0136] S403 inputs user basic features, user historical behavior features, advertising creative basic features, advertising creative playback features, and cross features into the FM model for training.

[0137] In some embodiments, Figure 5 This is a schematic diagram of the advertising material scoring model provided in an embodiment of this application. See also... Figure 5 As shown, the user's basic features and historical behavior features are combined to form user features, which are then input into the advertising creative scoring model. Similarly, the advertising creative's basic features and playback features are combined to form material features, which are also input into the advertising creative scoring model. The cross-features between the advertising creative and the user are input into the embedding layer 51 of the advertising creative scoring model for feature extraction. The embedded layer 51 then inputs the processed data into the feature cross-processing layer 52 for feature cross-processing. After the feature cross-processing is completed, the data is input into the sigmoid layer 53, which performs prediction scoring on the data and outputs the first predicted score.

[0138] In some embodiments, the expression (7) for the processing of input data by the sigmoid layer in the advertising creative scoring model is as follows:

[0139]

[0140] In equation (7), y j Indicates user u i For advertising materials i The first prediction score, where n represents the number of features input to the sigmoid layer, x i x represents the eigenvalue corresponding to the i-th feature; j w represents the feature value corresponding to the j-th feature, and w0 represents the preset bias value; w i The predefined feature x i The corresponding weight, w ij The predefined feature x i With feature x j Two-dimensional cross weights.

[0141] S304, the training device adjusts the parameters of the advertising material scoring model based on the first predicted score corresponding to the advertising material in the training samples and the positive and negative sample labels of the training samples until the convergence condition is met, and obtains the trained advertising material scoring model.

[0142] Understandably, after obtaining the first predicted score for each ad creative, the actual click behavior of users who actually clicked on the ad creative is obtained. By comparing the first predicted score with the actual click behavior, the parameters of the ad creative scoring model are adjusted based on the comparison results until the trained ad creative scoring model is obtained.

[0143] This application provides a method for predicting advertising creatives. Figure 6 This is an optional flowchart illustrating the advertising material prediction method provided in this application embodiment. See also... Figure 6 As shown, the above method may include:

[0144] S601, obtain the basic characteristics and playback characteristics of the target ad creative to be predicted, the basic characteristics and historical behavior characteristics of multiple active users corresponding to the ad slot to which the target ad creative belongs, and the cross characteristics between the target ad creative and each active user.

[0145] Among them, the multiple active users corresponding to the ad slot refer to the set of users who have made a call to any ad material of the ad slot within a preset time period.

[0146] In some embodiments, the sample active users are a set of users who have clicked on the ad creative of the ad placement within a preset time period. For example, the sample active users {u1, u2, ..., u...} m} represents the m users who clicked on the ad creative of ad slot A in the past 4 days.

[0147] S602, the advertising creative's basic features and playback features, the basic features and historical behavior features of multiple active users, and cross-features are input into the trained advertising creative scoring model for prediction processing, resulting in multiple second predicted scores corresponding to the target advertising creative. The advertising creative scoring model is trained using the aforementioned model training method, and the second predicted scores are used to characterize the degree of interest of active users in the target advertising creative.

[0148] S603, based on multiple second prediction scores corresponding to the target ad creative, determine the total predicted score of the target ad creative corresponding to multiple active users, and rank the target ad creative based on the total predicted score.

[0149] Understandably, once the advertising creative scoring model is trained, it will be deployed online. During online use, after obtaining multiple second-prediction scores corresponding to the target advertising creative, the target advertising creative can be sorted.

[0150] In some possible implementations, S603 may include: summing multiple second predicted scores corresponding to the target advertising material to obtain a total predicted score for the target advertising material corresponding to multiple active users; determining a first difference as the difference between the total predicted score of the target advertising material and the minimum value of the multiple second predicted scores; determining a second difference as the difference between the maximum value of the multiple second predicted scores and the minimum value of the multiple second predicted scores; determining the ratio of the first difference to the second difference as the standardized predicted score of the target advertising material; and sorting the target advertising material based on the standardized predicted score of the target advertising material.

[0151] Understandably, the multiple second prediction scores and total prediction scores corresponding to the target ad creative can be normalized to obtain a standardized prediction score for each target ad creative.

[0152] In some embodiments, taking the number of active users in the sample as n as an example, the target advertising creatives s i For each of the n second-prediction scores, the target ad creative s i The second set of predicted scores can be represented as {y1, y2, ..., y n}, calculate target ad creatives s i The expression for the predicted total score (8) is as follows:

[0153]

[0154] In equation (8), Y i Indicates target ad creatives s i The predicted total score, yj is the score of the j-th active user for the target ad creative s. i The first predicted score.

[0155] In some embodiments, the expression (9) for calculating the standardized prediction score for each target ad creative is as follows:

[0156] Y ave =(Y i -Y min ) / (Y max -Y min (9)

[0157] In equation (9), Y av e represents the target advertising material. i The corresponding standardized prediction score, Y i Indicates target ad creatives s i The predicted total score, Y min Indicates target ad creatives s i The minimum value among the corresponding multiple second prediction scores, Y max Indicates target ad creatives si The maximum value among the corresponding multiple second prediction scores.

[0158] In some embodiments, after determining the standardized prediction score corresponding to each target ad creative, the target ad creatives are ranked based on the standardized prediction scores, and the target ad creatives can be placed as high-quality ad creatives in the ad slots of the next time period.

[0159] In this embodiment of the application, after the advertising material scoring model outputs the second predicted score, the second predicted scores corresponding to each target advertising material are aggregated and used as the total predicted score of the target advertising material for sorting, which can facilitate the differentiation of the quality of advertising materials.

[0160] In this embodiment, positive and negative samples are sampled based on the exposure logs and call logs of the advertising platform and a preset overall sampling ratio. This enables dynamic sampling at the creative granularity level based on the exposure and click data of the advertising creative, optimizes the negative sample sampling method, and improves the quality of both positive and negative samples. The advertising creative scoring model trained based on this can achieve accurate targeting of high-quality creatives.

[0161] Furthermore, adding ad placements as a sampling dimension during the sampling process can more closely reflect programmatic customer acquisition scenarios; and from the perspective of material granularity, a higher-quality database can be built.

[0162] Furthermore, in conjunction with actual programmatic customer acquisition business scenarios, playback features that can effectively evaluate video quality have been added. Utilizing existing caller data, a matching score for each set of ad creatives is calculated by combining interest tags, tag scores, and duration coefficients, thus constructing user historical behavior features that reflect user interests and preferences. The introduction of cross-features increases the overall feature dimensionality of the ad creative scoring model and improves its ability to fit complex data.

[0163] The following continues to describe an exemplary structure in which the training device 455 for the model provided in the embodiments of this application is implemented as a software module. In some embodiments, such as Figure 2As shown, the software modules in the training device 455 of the advertising creative scoring model stored in the memory 450 may include: a sampling module 4551, used to determine a training sample set based on a set of advertising creatives with caller records; the caller records are used to record the caller behavior of users clicking on advertising creatives exposed on the advertising platform to enter the advertiser's designated page; the training sample set includes multiple training samples corresponding to each advertising creative, wherein the positive samples in the multiple training samples are matching pairs between advertising creatives and corresponding clicking users, and the negative samples in the multiple training samples are matching pairs between advertising creatives and non-clicking users, where clicking users refer to users who have caller behavior for advertising creatives, and non-clicking users refer to users who have no caller behavior for advertising creatives but have caller behavior for other advertising creatives in the same advertising slot; and a data processing module 4552, used to determine the training sample set. The initial advertising material scoring model is composed of the following components: basic features of the advertising material, advertising material playback features, basic user features, user historical behavior features, and cross-features between the advertising material and the user. The scoring training module 4553 takes these components and inputs them into the initial advertising material scoring model for prediction processing, obtaining the first predicted score for the advertising material in the training sample. The first predicted score represents the degree of user interest in the advertising material in the training sample. The training module 4554 adjusts the parameters of the advertising material scoring model based on the first predicted score and the positive and negative labels of the training samples until convergence is achieved, resulting in a trained advertising material scoring model.

[0164] In some possible implementations, the sampling module 4551 is used to determine, based on the call logs of the advertising platform, a set of advertising creatives with call records within a preset first time period, and positive samples corresponding to each advertising creative in the set of advertising creatives; wherein, the call logs include all call records in the advertising platform, and the number of positive samples corresponding to the advertising creatives is equal to the number of users who have made call behavior to the advertising creatives; and to determine, based on the exposure logs, call logs, and a preset overall sampling ratio of the advertising platform, negative samples corresponding to each advertising creative in the set of advertising creatives; wherein, the exposure logs include the exposure records of each advertising creative in the advertising platform, the overall sampling ratio is the ratio between positive and negative samples set for the ad placement, and the number of negative samples corresponding to the advertising creatives is determined based on the exposure volume, call volume, and overall sampling ratio of the advertising creatives.

[0165] In some possible implementations, the sampling module 4551 is used to determine, based on the call logs of the advertising platform, a set of advertising creatives with call records within a preset first time period, and positive samples corresponding to each advertising creative in the set of advertising creatives; wherein, the call logs include all call records in the advertising platform, and the number of positive samples corresponding to the advertising creatives is equal to the number of users who have made call behavior to the advertising creatives; and to determine, based on the exposure logs, call logs, and a preset overall sampling ratio of the advertising platform, negative samples corresponding to each advertising creative in the set of advertising creatives; wherein, the exposure logs include the exposure records of each advertising creative in the advertising platform, the overall sampling ratio is the ratio between positive and negative samples set for the ad placement, and the number of negative samples corresponding to the advertising creatives is determined based on the exposure volume, call volume, and overall sampling ratio of the advertising creatives.

[0166] In some possible implementations, the data processing module 4552 is used to obtain the basic user features of users in the training samples from the user feature library of the material recommendation system; obtain the basic advertising material features of advertising materials in the training samples from the material library of the material recommendation system; obtain the user historical behavior features of users in the training samples from the call logs of the advertising platform; obtain the advertising material playback features of advertising materials in the training samples from the full-site video playback database of the material recommendation system; and determine the cross features between users and advertising materials in the training samples based on the user information of users in the training samples and the material information of advertising materials.

[0167] In some possible implementations, the data processing module 4552 is used to obtain multiple historical advertising materials clicked by users in the training samples within a preset second time period from the call logs; determine the duration coefficient of the multiple historical advertising materials based on the call logs; the duration coefficient is used to represent the difference between the month in which the historical advertising material was published and the current month; obtain the interest tags and tag scores corresponding to each historical advertising material; divide the multiple historical advertising materials into multiple groups of historical advertising materials according to the interest tags; each group of historical advertising materials corresponds to one interest tag; calculate the user's matching score for each group of historical advertising materials according to the tag score and duration coefficient corresponding to each historical advertising material; sort the multiple matching scores, and use the multiple groups of historical advertising materials whose matching scores meet a preset threshold as the user's historical behavior features.

[0168] In some possible implementations, playback features include one or any combination of the following: actual play count, actual play rate, average play rate, x% completion rate, and completion rate; wherein, actual play count is used to characterize the number of times the ad creative is played; actual play rate is used to characterize the ratio between the number of times the ad creative is played and the number of times the ad creative is exposed; average play rate is used to characterize the average of the actual play rates of all ad creatives in the same ad slot; x% completion rate is used to characterize the ratio between the number of times the user plays the ad creative for the duration to the full duration of the ad creative is greater than x% and the actual play count; completion rate represents the ratio between the number of times the user fully plays the ad creative and the actual play count. The data processing module 4552 is used to obtain the actual play count and exposure count corresponding to the advertising materials in the training samples from the site's video playback database; based on the actual play count and exposure count corresponding to the advertising materials; calculate the actual play rate and average play rate; obtain the playback duration corresponding to the advertising materials from the site's video playback database; when the advertising materials are played completely, calculate the ratio of the number of times the advertising materials are played completely to their corresponding actual play count to obtain the completion rate of the advertising materials; when the advertising materials are not played completely, calculate the ratio of the number of times the playback duration of the advertising materials is greater than x% to their corresponding actual play count to obtain the x% completion rate of the advertising materials.

[0169] In some possible implementations, the cross features include type cross features; the data processing module 4552 is used to determine the first tag corresponding to the user and the second tag corresponding to the advertising material based on the user information of the user and the material information of the advertising material in the training samples; if the first tag and the second tag are the same, the type cross feature between the user corresponding to the first tag and the advertising material corresponding to the second tag is determined as the first type cross feature; if the first tag and the second tag are different, the type cross feature between the user corresponding to the first tag and the advertising material corresponding to the second tag is determined as the second type cross feature.

[0170] In some possible implementations, the cross features further include a first numerical cross feature and a second numerical cross feature; the first label includes multiple first interest labels, and the second interest label includes multiple second interest labels; the data processing module 4552 is used to determine multiple first interest labels corresponding to users in the training samples, and to determine multiple second interest labels corresponding to advertising materials in the training samples; to perform matching processing on the multiple first interest labels of users and the multiple second interest labels of advertising materials, and to determine the number of mutually matching first interest labels and second interest labels as the first numerical cross feature between users and advertising materials; when there is a first numerical cross feature between users and advertising materials, to obtain preset interest label weights; one first interest label corresponds to one interest label weight; and to determine the second numerical cross feature between users and advertising materials based on the first numerical cross feature and the interest label weights.

[0171] The following continues to describe the exemplary structure of the advertising material prediction device 456 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules in the advertising material prediction device 456 stored in the memory 450 may include: a data acquisition module 4561, used to acquire the basic features and playback features of the target advertising material to be predicted, the basic features and historical behavior features of multiple active users corresponding to the ad slot to which the target advertising material belongs, and the cross features between the target advertising material and each active user; wherein, the multiple active users corresponding to the ad slot refer to the set of users who have call-to-the-end behavior for any advertising material of the ad slot within a preset time period; a scoring module 4562, used to input the basic features and playback features of the target advertising material, the basic features and historical behavior features of multiple active users, and the cross features into the trained advertising material scoring model for prediction processing, to obtain multiple second predicted scores corresponding to the target advertising material; the advertising material scoring model is trained by the above model training method, and the second predicted scores are used to characterize the degree of interest of active users in the target advertising material; a ranking module 4563, used to determine the total predicted score of the target advertising material corresponding to multiple active users based on the multiple second predicted scores corresponding to the target advertising material, so as to rank the target advertising material based on the total predicted score.

[0172] In some possible implementations, the sorting module 4563 is used to sum and calculate multiple second predicted scores corresponding to the target advertising material to obtain the total predicted score of the target advertising material corresponding to multiple active users; determine the difference between the total predicted score of the target advertising material and the minimum value of the multiple second predicted scores as a first difference; determine the difference between the maximum value of the multiple second predicted scores and the minimum value of the multiple second predicted scores as a second difference; determine the ratio of the first difference to the second difference as the standardized predicted score of the target advertising material; and sort the target advertising material based on the standardized predicted score of the target advertising material.

[0173] This application provides a computer program product or computer program that includes computer instructions stored in a computer storage medium. A processor of a computer device reads the computer instructions from the computer storage medium and executes the computer instructions, causing the computer device to perform the model training method and advertising material prediction method described above in this application.

[0174] This application provides a computer storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the model training method and advertising material prediction method provided in this application. Figure 3 The training method of the model shown and Figure 6 The advertising creative prediction method is shown.

[0175] In some embodiments, the computer storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0176] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0177] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0178] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0179] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for training a model, characterized in that, include: Based on the set of advertising creatives with call records, a training sample set is determined; The call log is used to record the call behavior of users clicking on the advertiser's designated page through the ad creative exposed on the ad placement platform; the training sample set includes multiple training samples corresponding to each ad creative, wherein the positive samples in the multiple training samples are the matching pairs between the ad creative and the corresponding clicking users, and the negative samples in the multiple training samples are the matching pairs between the ad creative and non-clicking users. The clicking user refers to the user who has a call behavior for the ad creative, and the non-clicking user refers to the user who has no call behavior for the ad creative but has a call behavior for other ad creatives in the same ad position; Determine the basic features of the advertising creative, the playback features of the advertising creative, the basic features of the user, the historical behavior features of the user, and the cross features between the advertising creative and the user corresponding to the training sample; The basic features of the advertising creatives, the playback features of the advertising creatives, the basic features of the users, the historical behavioral features of the users, and the cross features between the advertising creatives and the users corresponding to the training samples are input into the initial advertising creative scoring model for prediction processing to obtain the first prediction score corresponding to the advertising creatives in the training samples; the first prediction score is used to characterize the degree of interest of users in the training samples towards the advertising creatives; Based on the first predicted score corresponding to the advertising creative in the training samples and the positive and negative sample labels of the training samples, the parameters of the advertising creative scoring model are adjusted until the convergence condition is met, and the trained advertising creative scoring model is obtained.

2. The method according to claim 1, characterized in that, The training sample set is determined based on the set of advertising creatives with caller records, including: Based on the call logs of the advertising platform, determine the set of advertising creatives with call records within a preset first time period, and the positive samples corresponding to each advertising creative in the set of advertising creatives; wherein, the call logs include all call records in the advertising platform, and the number of positive samples corresponding to the advertising creative is equal to the number of users who have call behavior for the advertising creative; Based on the exposure logs, call logs, and preset overall sampling ratio of the advertising platform, the negative samples corresponding to each advertising material in the advertising material set are determined; wherein, the exposure logs include the exposure records of each advertising material in the advertising platform, the overall sampling ratio is the ratio between positive and negative samples set for the advertising position, and the number of negative samples corresponding to the advertising material is determined based on the exposure volume, call volume, and overall sampling ratio of the advertising material.

3. The method according to claim 2, characterized in that, The step of determining the negative sample corresponding to each advertising creative in the advertising creative set based on the exposure logs, call logs, and preset overall sampling ratio of the advertising platform includes: For any ad creative in the set of ad creatives, obtain the exposure volume of the ad creative and the total exposure volume of all ad creatives in the ad slot to which the ad creative belongs from the exposure log; Based on the call records, the number of calls to the advertising creative and the total number of calls to all advertising creatives in the advertising slot to which the advertising creative belongs are obtained; the number of calls is used to represent the number of users who have made calls to the advertising creative. Based on the overall sampling ratio, the total exposure of all advertising creatives, and the total number of calls to all advertising creatives, calculate the negative sample coefficient corresponding to the advertising creative; The difference between the exposure of the advertising material and the call volume of the advertising material is multiplied by the negative sample coefficient to obtain the number of negative samples corresponding to the advertising material. Based on the number of negative samples corresponding to the advertising creative, a corresponding number of non-clicking users of the advertising creative are selected from the negative sample candidate set of the advertising position to which the advertising creative belongs, to form the negative samples corresponding to the advertising creative; wherein, the negative sample candidate set is the set of users who have call behavior for any advertising creative in the same advertising position.

4. The method according to claim 1, characterized in that, The process of determining the basic features of the advertising creative, the playback features of the advertising creative, the basic features of the user, the historical behavior features of the user, and the cross-features between the advertising creative and the user corresponding to the training sample includes: In the user feature library of the material recommendation system, obtain the basic user features of the users in the training samples; In the material library of the material recommendation system, obtain the basic characteristics of the advertising materials in the training samples; In the call logs of the advertising platform, obtain the user history behavior characteristics of the users in the training samples; In the full-site video playback database of the material recommendation system, obtain the ad material playback characteristics of the ad materials in the training samples; Based on the user information of the users in the training samples and the material information of the advertising materials, the cross-features between the users and the advertising materials in the training samples are determined.

5. The method according to claim 4, characterized in that, The step of obtaining the user history behavior characteristics of the users in the training samples from the call logs of the advertising delivery platform includes: In the call log, obtain multiple historical advertising materials that the user in the training sample clicked within a preset second time period; Based on the call logs, the duration coefficients of the multiple historical advertising materials are determined; the duration coefficients are used to represent the difference between the month in which the historical advertising materials were published and the current month. Obtain the interest tags and tag scores corresponding to each historical ad creative; The multiple historical advertising materials are divided into multiple groups of historical advertising materials according to interest tags; each group of historical advertising materials corresponds to one interest tag. Based on the tag score and duration coefficient corresponding to each historical ad creative, the matching score of the user for each set of historical ad creatives is calculated; Multiple matching scores are sorted, and multiple sets of historical advertising materials whose matching scores meet a preset threshold are used as the user's historical behavior characteristics.

6. The method according to claim 4, characterized in that, The playback characteristics include one or any combination of the following: actual play count, actual play rate, average play rate, x% completion rate, and completion rate; wherein, the actual play count is used to characterize the number of times the ad creative is played; the actual play rate is used to characterize the ratio between the number of times the ad creative is played and the number of times the ad creative is exposed; the average play rate is used to characterize the average of the actual play rates of all ad creatives in the same ad slot; the x% completion rate is used to characterize the ratio between the number of times the user plays the ad creative for the duration to the total duration of the ad creative is greater than x% and the actual play count; the completion rate represents the ratio between the number of times the user fully plays the ad creative and the actual play count; The step of obtaining the ad creative playback features of the ad creatives in the training samples from the full-site video playback database of the material recommendation system includes: In the site-wide video playback database, obtain the actual play count and exposure count corresponding to the advertising materials in the training samples; Based on the actual play count and exposure count corresponding to the advertising material, calculate the actual play rate and the average play rate; Obtain the playback duration corresponding to the advertising material from the site-wide video playback database; If the advertising material is played in its entirety, the completion rate of the advertising material is obtained by calculating the ratio of the number of times the advertising material is played in its entirety to its corresponding actual number of views. If the advertising material is not fully played, the x% completion rate of the advertising material is obtained by calculating the ratio of the number of times the playback duration of the advertising material is greater than x% to the corresponding actual number of plays.

7. The method according to claim 4, characterized in that, The cross features include type cross features; The step of determining the cross-features between users and advertising materials in the training samples based on user information and advertising material information includes: Based on the user information of the users and the material information of the advertising materials in the training samples, determine the first tag corresponding to the user and the second tag corresponding to the advertising materials; When the first tag and the second tag are the same, the type cross feature between the user corresponding to the first tag and the advertising material corresponding to the second tag is determined as the first type cross feature; If the first tag and the second tag are different, the type cross feature between the user corresponding to the first tag and the advertising material corresponding to the second tag is determined as the second type cross feature.

8. The method according to claim 7, characterized in that, The cross features also include a first numerical cross feature and a second numerical cross feature; the first label includes multiple first interest labels, and the second label includes multiple second interest labels; The step of determining the cross-features between users and advertising materials in the training samples based on user information and advertising material information includes: Determine multiple first interest tags corresponding to users in the training sample, and determine multiple second interest tags corresponding to advertising materials in the training sample; The user's multiple first interest tags and the advertising material's multiple second interest tags are matched, and the number of mutually matching first interest tags and second interest tags is determined as the first numerical cross feature between the user and the advertising material; When there is a first numerical cross feature between the user and the advertising material, a preset interest tag weight is obtained; one first interest tag corresponds to one interest tag weight. Based on the first numerical cross feature and the interest tag weight, a second numerical cross feature between the user and the advertising material is determined.

9. A method for predicting advertising creatives, characterized in that, include: The process involves obtaining the basic characteristics and playback characteristics of the target advertising creative to be predicted, the basic user characteristics and historical behavior characteristics of multiple active users corresponding to the ad slot to which the target advertising creative belongs, and the cross-features between the target advertising creative and each active user; wherein, the multiple active users corresponding to the ad slot refer to the set of users who have made call-to-the-end behavior for any advertising creative in the ad slot within a preset time period. The advertising material scoring model trained by inputting the basic features and playback features of the target advertising material, the basic features and historical behavior features of the multiple active users, and the cross-features are subjected to prediction processing to obtain multiple second prediction scores corresponding to the target advertising material; the advertising material scoring model is trained by the method described in any one of claims 1 to 8, and the second prediction scores are used to characterize the degree of interest of the active users in the target advertising material; Based on multiple second predicted scores corresponding to the target ad creative, the predicted total score of the target ad creative corresponding to the multiple active users is determined, and the target ad creative is sorted based on the predicted total score.

10. The method according to claim 9, characterized in that, The step of determining the predicted total score of the target advertising creative corresponding to the multiple active users based on the multiple second predicted scores, and then ranking the target advertising creative based on the predicted total score, includes: The summation of multiple second prediction scores corresponding to the target advertising creative is performed to obtain the total prediction score of the target advertising creative for the multiple active users; The difference between the predicted total score of the target advertising material and the minimum value of the plurality of second predicted scores is determined as the first difference; The difference between the maximum value of the plurality of second predicted scores and the minimum value of the plurality of second predicted scores is determined as the second difference; The ratio of the first difference to the second difference is determined as the standardized prediction score of the target advertising material; The target advertising creatives are ranked based on their standardized prediction scores.

11. An advertising material prediction device, characterized in that, include: The data acquisition module is used to acquire the basic characteristics and playback characteristics of the target advertising material to be predicted, the basic user characteristics and historical user behavior characteristics of multiple active users corresponding to the advertising slot to which the target advertising material belongs, and the cross-features between the target advertising material and each active user; wherein, the multiple active users corresponding to the advertising slot refer to the set of users who have call-to-the-end behavior for any advertising material of the advertising slot within a preset time period. The scoring module is used to perform prediction processing on the advertising material scoring model trained by inputting the basic features and playback features of the target advertising material, the basic features and historical behavior features of the multiple active users, and the cross-features, to obtain multiple second predicted scores corresponding to the target advertising material; the advertising material scoring model is trained by the method described in any one of claims 1 to 8, and the second predicted scores are used to characterize the degree of interest of the active users in the target advertising material; The sorting module is used to determine the predicted total score of the target advertising material corresponding to the multiple active users based on the multiple second predicted scores corresponding to the target advertising material, so as to sort the target advertising material based on the predicted total score.

Images