Training methods, devices, equipment, and media for click-through rate prediction models

Abstract

This application discloses a training method, apparatus, device, and medium for a click-through rate (CTR) prediction model, belonging to the field of artificial intelligence. The method includes: obtaining historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences; inputting the historical sample behavior sequences and current sample click items into a CTR prediction model to predict a first probability that the current sample click item will be clicked at a reference time; inputting the historical sample behavior sequences and future sample click items into the CTR prediction model to predict a second probability that the future sample click item will be clicked after the reference time; training the CTR prediction model based on the error between the probability label of the current sample click item and the first probability; and training the CTR prediction model based on the error between the probability label of the future sample click item and the second probability. This method improves the utilization rate of information in user sample behavior sequences.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, device, and medium for training a click-through rate prediction model. Background Technology

[0002] Click-through rate (CTR) prediction models aim to predict a user's click-through rate on a particular item at the next moment based on the user's historical behavior sequence.

[0003] In related technologies, when training a click-through rate (CTR) prediction model, a complete sequence of sample behaviors is used to supervise the CTR prediction model's CTR in the next clicked item. The complete sequence of sample behaviors includes sample clicked items at multiple time points arranged in chronological order.

[0004] Training a click-through rate prediction model using related technologies wastes other information contained in the complete sample behavior sequence. Summary of the Invention

[0005] This application provides a training method, apparatus, device, and medium for a click-through rate (CTR) prediction model, which improves the utilization rate of information in user sample behavior sequences. The technical solution is as follows:

[0006] According to one aspect of this application, a method for training a click-through rate (CTR) prediction model is provided, the method comprising:

[0007] Obtain historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences; the user sample behavior sequence includes multiple sample click items arranged in chronological order, the historical sample behavior sequence includes multiple sample click items located before the reference time, the current sample click item is the sample click item corresponding to the reference time, and the future sample click item is the sample click item located after the reference time;

[0008] By inputting the historical sample behavior sequence and the current sample clicked item into the click-through rate prediction model, the first probability of the current sample clicked item being clicked at the reference time is predicted.

[0009] By inputting historical sample behavior sequences and future sample click items into the click-through rate prediction model, the second probability of future sample click items being clicked after the reference time is predicted.

[0010] A click-through rate (CTR) prediction model is trained based on the error between the probability label of the currently clicked item and the first probability; and a CTR prediction model is trained based on the error between the probability label of the future clicked item and the second probability.

[0011] According to another aspect of this application, a training apparatus for a click-through rate prediction model is provided, the apparatus comprising:

[0012] The acquisition module is used to acquire historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences. The user sample behavior sequence includes multiple sample click items arranged in chronological order, the historical sample behavior sequence includes multiple sample click items located before the reference time, the current sample click item is the sample click item corresponding to the reference time, and the future sample click item is the sample click item located after the reference time.

[0013] The prediction module is used to input the historical sample behavior sequence and the current sample clicked item into the click rate prediction model to predict the first probability that the current sample clicked item will be clicked at a reference time.

[0014] The prediction module is also used to input historical sample behavior sequences and future sample click items into the click-through rate prediction model to predict the second probability that future sample click items will be clicked after the reference time.

[0015] The training module is used to train a click-through rate prediction model based on the error between the probability label of the currently clicked item and the first probability; and to train a click-through rate prediction model based on the error between the probability label of the future clicked item and the second probability.

[0016] According to one aspect of this application, a computer device is provided, comprising: a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the training method of the click-through rate prediction model as described above.

[0017] According to another aspect of this application, a computer-readable storage medium is provided, which stores a computer program that is loaded and executed by a processor to implement the training method of the click-through rate prediction model as described above.

[0018] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described above.

[0019] The beneficial effects of the technical solutions provided in this application include at least the following:

[0020] By acquiring historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences, a click-through rate (CTR) prediction model is trained based on both historical and future sample click items. This method improves the utilization rate of information from user sample behavior sequences.

[0021] Compared to related technologies, this application not only performs the task of predicting the clicked items of the current sample, but also the task of predicting the clicked items of future samples. Historical sample behavior sequences implicitly contain user preference features. This application takes into account that user preference features are related not only to the clicked items of the current sample, but also to the clicked items of future samples.

[0022] In this application, future sample click items will be used as enhanced positive samples for training, thus improving the training data. This data augmentation of the training samples can alleviate the sparsity problem of user click items in real-world usage, as well as the variability and irregularity of user click behavior. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of a principle provided by an exemplary embodiment of this application;

[0025] Figure 2 This is a flowchart of a training method for a click-through rate prediction model provided in one embodiment of this application;

[0026] Figure 3 This is a schematic diagram of another exemplary embodiment provided in this application;

[0027] Figure 4 This is a flowchart of a training method for a click-through rate prediction model provided in another embodiment of this application;

[0028] Figure 5 This is a schematic diagram of another exemplary embodiment provided in this application;

[0029] Figure 6 This is a flowchart of a training method for a click-through rate prediction model provided in another embodiment of this application;

[0030] Figure 7 This is a schematic diagram of another exemplary embodiment provided in this application;

[0031] Figure 8 This is a structural block diagram of a training device for a click-through rate prediction model provided in one embodiment of this application;

[0032] Figure 9 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0034] First, a brief introduction to the terms used in the embodiments of this application:

[0035] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0036] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0037] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0038] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0039] Knowledge distillation (KD) generally utilizes the knowledge learned by the large model to guide the training of the smaller model. This allows the smaller model to achieve performance comparable to the large model, but with a significantly reduced number of parameters, thus compressing and accelerating the model. This is the application of knowledge distillation and transfer learning in model optimization. In related technologies, knowledge distillation adopts a teacher-student model. The complex and large model acts as the teacher, while the student model has a simpler structure. The teacher model assists in training the student model. The teacher model, with its strong learning ability, can transfer its knowledge to the relatively weaker student model, thereby enhancing the student model's generalization ability. In real-world applications, the complex and cumbersome but effective teacher model remains offline, simply acting as a mentor. The flexible and lightweight student model is the one actually deployed for prediction tasks. Knowledge distillation is the transfer of a model's capabilities. Based on different transfer methods, it can be broadly divided into two directions: target-based distillation (also known as soft-target distillation or Logits method distillation) and feature-based distillation algorithms.

[0040] Self-distillation (SD) employs a supervised learning approach to knowledge distillation. In SD, the teacher and student models are a single model, guiding the learning process and completing the knowledge distillation.

[0041] Sequential Recommendation (SR) aims to provide users with suitable items to click based on the order of their historical behavior. Related sequential recommendation models are trained under the task of predicting the next clicked item, thus facing two challenges: the sparsity of user feedback data and the variability and irregularity of user click behavior. Unlike natural language in Natural Language Processing (NLP), the behavioral sequences in sequential recommendation are more personalized, irregular, and disordered. Therefore, the current user preference extracted from historical behavior may be related not only to the next-1 (current clicked) item to be predicted but also to the next-k (future clicked) item. Inspired by this phenomenon, this application proposes a novel self-distilled future information gain model (FASRec) for recommendation. Hereinafter, this future information gain model is also commonly referred to as a sequential recommendation model or a click-through rate prediction model.

[0042] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the user sample behavior sequences and sample click items involved in this application were obtained with full authorization.

[0043] Furthermore, when dealing with relevant information, the relevant information processor will adhere to the principles of legality, legitimacy, and necessity, clearly define the purpose, method, and scope of the relevant information processing, obtain the consent of the relevant information subject, and take necessary technical and organizational measures to ensure the security of the relevant information.

[0044] Figure 1 A schematic diagram illustrating an exemplary embodiment of this application is shown. Figure 1 A click-through rate prediction model trained by a computer device is shown. Figure 1 A training architecture 100 for a click-through rate (CTR) prediction model is provided. The training architecture 100 is used to train the CTR prediction model 101.

[0045] Obtain user sample behavior sequences, which include multiple sample click items arranged in chronological order. Figure 1 The user sample behavior sequence shown includes sample click items (V1, V2…V) from time 1 to time t+n. t+m V t+n ).

[0046] Obtain the historical sample behavior sequence 102, the current sample click item 103, and the future sample click item 106 from the user sample behavior sequence. Figure 1 The historical sample behavior sequence 102 shown includes sample click items (V1, V2...Vt) from time 1 to time t. t The current sample clicked item 103 is the sample clicked item at time t+1 (V t+1 ). Future sample click item 106 refers to the sample click item after time t+1 in the user sample behavior sequence (i.e., V). t+2 To V t+n One of the sample clicked items. Figure 1 For V t+m ).

[0047] Figure 1 The click-through rate prediction model 101 to be trained is also shown. Figure 1 In the training architecture shown, the historical sample behavior sequence 102 and the current sample clicked item 103 are input into the click-through rate prediction model 101 to predict the first probability 104 that the current sample clicked item 103 is clicked at time t+1. The click-through rate prediction model 101 is trained based on the first probability 104 and the probability label 105 of the current sample clicked item 103.

[0048] Figure 1 The training architecture also shows that by inputting the historical sample behavior sequence 102 and the future sample click item 106 into the click-through rate prediction model 101, a second probability 107 is predicted that the future sample click item 106 will be clicked after time t+1. Based on the second probability 107 and the probability label 108 of the future sample click item, the click-through rate prediction model 101 is trained.

[0049] Reference Figure 1 The training method for the click-through rate prediction model provided by related technologies only performs the task of predicting the clicked item 103 in the current sample. Combined with... Figure 1 The training architecture shown in this application not only performs the task of predicting the clicked item 103 of the current sample, but also the task of predicting the clicked item 106 of future samples. The historical sample behavior sequence 102 contains implicit user preference features. Figure 1 The training architecture not only takes into account the correlation between user preference features and the current sample click item 103 to be predicted, but also the correlation between the future sample click item 106.

[0050] Understandably, future sample click item 106 will be used as an enhanced positive sample for training, supplementing the current sample click item 103. By augmenting the training samples, the problem of data sparsity in user click items during real-world use can be alleviated, as well as the variability and irregularity of user click behavior in real-world usage.

[0051] In some embodiments, the click-through rate (CTR) prediction model is trained by a server and / or a terminal. The CTR prediction model can be trained by only one or more servers, only one or more terminals, or jointly trained by at least one server and at least one terminal. The aforementioned servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The aforementioned terminals can be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, etc., but are not limited to these. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0052] Figure 2 A flowchart illustrating a training method for a click-through rate prediction model provided in an exemplary embodiment of this application is shown, wherein the method is... Figure 1 The computer device shown is used for illustrative purposes. The method includes:

[0053] Step 210: Obtain historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences;

[0054] User sample behavior sequence: refers to the click behavior sequence of a single user in the training dataset. The user sample behavior sequence includes multiple sample click items arranged in chronological order. For example, the user sample behavior sequence includes sample click items V1 at time t, V2 at time t, ..., and Vn at time t+n, arranged in chronological order. t+n Clickable items refer to items on the terminal interface that users can click. Examples include advertisements, news items, headlines, events, and entity words.

[0055] Historical sample behavior sequence: This includes multiple sample click items in the user sample behavior sequence that are prior to a reference time. Optionally, the reference time is time t+1. The historical sample behavior sequence includes t sample click items in the user sample behavior sequence that are prior to time t+1 (i.e., the t sample click items corresponding to the previous t times). For example, the historical sample behavior sequence includes sample click items V1 at the first time, V2 at the second time, ..., and V at time t, arranged in sequence. t t is a positive integer greater than or equal to 1.

[0056] Current sample click item: refers to the sample click item in the user sample behavior sequence corresponding to the reference time. Optionally, the reference time is time t+1, which can also be called the current time. The current sample click item can also be represented as V. t+1 .

[0057] Future sample click items: These refer to sample click items in the user sample behavior sequence that occur after a reference time. Optionally, the reference time is time t+1. In other words, future sample click items are sample click items that occur after the current sample click item. For example, future sample click items can be represented as a user sample behavior sequence (V1, V2, ... V...). t+1 V t+m V t+n V in ) t+m Where m is greater than 1 and m is less than n.

[0058] In one embodiment, from user sample behavior sequences (V1, V2, ... V... t+1 V t+m V t+n The item V in the current sample is located in the middle. t+1 In subsequent sample click projects, future sample click projects V are randomly selected. t+m Optional, from V t+1 Subsequent sample click items are randomly sampled with equal probability (1 / (nt)) to obtain future sample click items V. t+m .

[0059] Step 220: Input the historical sample behavior sequence and the current sample clicked item into the click rate prediction model to predict the first probability that the current sample clicked item will be clicked at the reference time.

[0060] In one embodiment, the historical sample behavior sequence (V1, V2…V) is used. t ) and the current sample clicked item (V t+1 Input the click-through rate (CTR) prediction model, and the CTR prediction model will output the click value (V) of the current sample at the next time step (i.e., time t+1). t+1 The first probability of ).

[0061] In one embodiment, the click-through rate (CTR) prediction model is any one of GRU4Rec, BERT4Rec, SASRec, and CL4Srec. GRU4Rec, BERT4Rec, SASRec, and CL4Srec are all CTR prediction models provided by related technologies. Therefore, it can be understood that this application provides a general training architecture, and any CTR prediction model from related technologies can be trained using the method described in this application.

[0062] Step 230: Input the historical sample behavior sequence and the future sample click item into the click rate prediction model to predict the second probability that the future sample click item will be clicked after the reference time.

[0063] In one embodiment, the historical sample behavior sequence (V1, V2…V) is used. t ) and future sample click projects (V t+m Input the click-through rate (CTR) prediction model, and the CTR prediction model will output the clicked items (V) of future samples after time t+1. t+m The second probability of ).

[0064] Step 240: Train the click-through rate prediction model based on the error between the probability label of the clicked item in the current sample and the first probability.

[0065] The probability label of the currently clicked item indicates the probability of the currently clicked item being clicked at a reference time. Optionally, the reference time is time t+1. Optionally, the probability label includes hard labels and / or soft labels. A hard label is a one-hot label, indicating that the proportion of positive samples in the currently clicked item is 1. For example, if the currently clicked item is product A, the hard labels for product A include 100% of the ballpoint pen category (positive sample category) and 0% of the non-ballpoint pen category (negative sample category), that is, the hard label is a binary label.

[0066] Soft labels indicate the percentage (p) of positive categories in the currently clicked item. For example, if the currently clicked item is product A, the soft labels for product A include 70% ballpoint pens (positive category), 20% pencils (negative category), and 10% fountain pens (negative category). In other words, soft labels are not binary labels.

[0067] In one embodiment, the hard label of the currently clicked item is used as the probability label of the currently clicked item. Since the currently clicked item is the clicked item in the user's sample behavior sequence corresponding to time t+1, the probability label of the currently clicked item is 1.

[0068] In one embodiment, the click-rate prediction model is trained based on the error between the first probability predicted by the click-rate prediction model and the probability label (hard label: 1) of the currently clicked item.

[0069] Step 250: Train the click-through rate prediction model based on the error between the probability label of the future sample clicked item and the second probability.

[0070] The probability label for a future sample click item indicates the probability that the item will be clicked after a reference time. Optionally, the reference time is time t+1. Optionally, the probability label includes hard labels and / or soft labels. A hard label is a one-hot label, indicating that the proportion of positive samples in the future sample click item is 1. For example, if the future sample click item is animal A, the hard label for animal A includes 100% dog category (positive sample category) and 0% non-dog category (negative sample category), that is, the hard label is a binary label.

[0071] Soft labels indicate the percentage (p) of positive categories in a future sample's clicked item. For example, if a future sample clicks on animal A, the soft labels for animal A would include 80% dog category (positive category), 10% wolf category (negative category), and 10% cat category (negative category). In other words, soft labels are not binary classification labels.

[0072] In one embodiment, the hard label of the future sample click item is used as the probability label of the future sample click item. Since the future sample click item is the sample click item located after time t+1 in the user sample behavior sequence, the probability label of the future sample click item is 1.

[0073] In one embodiment, the soft label of the future sample clicked item is used as the probability label of the future sample clicked item. Therefore, the probability label of the future sample clicked item is p.

[0074] In one embodiment, new labels are generated based on hard and soft labels. These new labels are then used as probability labels for future sample clicks on items.

[0075] In one embodiment, the click-through rate prediction model is trained based on the error between the second probability predicted by the click-through rate prediction model and the probability label of the clicked item in the future sample.

[0076] In summary, by acquiring historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences, a click-through rate (CTR) prediction model is trained based on the historical sample behavior sequences and current sample click items; a CTR prediction model is also trained based on the historical sample behavior sequences and future sample click items.

[0077] Compared to related technologies, this application not only performs the task of predicting the clicked items of the current sample, but also the task of predicting the clicked items of future samples. Historical sample behavior sequences implicitly contain user preference features. This application takes into account that user preference features are related not only to the clicked items of the current sample, but also to the clicked items of future samples.

[0078] In this application, future sample click items will be used as enhanced positive samples for training, thus improving the training data. This data augmentation of the training samples can alleviate the sparsity problem of user click items in real-world usage, as well as the variability and irregularity of user click behavior.

[0079] Figure 3 A schematic diagram illustrating an exemplary embodiment of this application is shown. Figure 3 The training architecture of the click-through rate prediction model is shown. Figure 3 A knowledge distillation method is used to train the click-through rate prediction model. Figure 3 The student click-through rate prediction model 301 (i.e., the student model) and the teacher click-through rate prediction model 302 (i.e., the teacher model) in the knowledge distillation architecture are shown. Figure 3 The purpose of the training architecture 300 is to train the student click-through rate prediction model 301.

[0080] Figure 3 This illustrates the acquisition of user sample behavior sequences (V1, V2, ..., V...). t ...V t+1 V t+m V t+n Obtain historical sample behavior sequences (V1, V2, ..., V) from user sample behavior sequences. t )303, Current sample clicked item 304 (V t+1 ) and future sample click item 305 (V t+m The historical sample behavior sequence 303 and the current sample clicked item 304 are input into the student click-through rate prediction model 301. The student click-through rate prediction model 301 will output the predicted first probability 306. Based on the first probability 306 and the hard label 307 of the current sample clicked item, a loss is constructed to train the student click-through rate prediction model 301.

[0081] Input the future sample click item 305 and the historical sample behavior sequence 303 into the student click rate prediction model 301. The student click rate prediction model 301 will output the predicted second probability 308. Based on the second probability 308, the soft label 309 of the future sample click item, and the hard label 310 of the future sample click item, construct a loss to train the student click rate prediction model 301.

[0082] Figure 3 It also shows how to obtain the soft label 309 of future sample click items. Figure 3 The diagram shows that inputting future sample click items 305 and historical sample behavior sequences 303 into the teacher click rate prediction model 302 outputs soft labels 309 for future sample click items.

[0083] Figure 4A flowchart illustrating a training method for a click-through rate prediction model provided in an exemplary embodiment of this application is shown, wherein the method is... Figure 1 The computer device shown is used to illustrate, for example, the method including:

[0084] Step 410: Obtain historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences;

[0085] User sample behavior sequence: refers to the click behavior sequence of a single user in the training dataset. The user sample behavior sequence includes multiple sample click items arranged in chronological order. For example, the user sample behavior sequence includes sample click items V1 at time t, V2 at time t, ..., and Vn at time t+n, arranged in chronological order. t+n Clickable items refer to items on the terminal interface that users can click. Examples include advertisements, news items, headlines, events, and entity words.

[0086] Historical sample behavior sequence: This includes multiple sample click items in the user sample behavior sequence that are prior to a reference time. Optionally, the reference time is time t+1. Optionally, the historical sample behavior sequence includes t sample click items in the user sample behavior sequence that are prior to time t+1 (i.e., t sample click items corresponding to the previous t times). For example, the historical sample behavior sequence includes sample click items V1 at the first time, V2 at the second time, ..., and V at time t, arranged in sequence. t t is a positive integer greater than or equal to 1.

[0087] Current sample click item: refers to the sample click item in the user sample behavior sequence corresponding to the reference time. Optionally, the reference time is time t+1, which can also be called the current time. The current sample click item can also be represented as V. t+1 .

[0088] Future sample click items: These refer to sample click items in the user sample behavior sequence that occur after a reference time. Optionally, the reference time is time t+1. In other words, future sample click items are sample click items that occur after the current sample click item. For example, future sample click items can be represented as a user sample behavior sequence (V1, V2, ... V...). t+1 V t+m V t+n V in ) t+m Where m is greater than 1 and m is less than n.

[0089] In one embodiment, from user sample behavior sequences (V1, V2, ... V... t+1 V t+m Vt+n The item V in the current sample is located in the middle. t+1 In subsequent sample click projects, future sample click projects V are randomly selected. t+m Optional, from V t+1 Subsequent sample click items are randomly sampled with equal probability (1 / (nt)) to obtain future sample click items V. t+m .

[0090] Step 420: Input the historical sample behavior sequence and the current sample clicked item into the student click rate prediction model to predict the first probability that the current sample clicked item will be clicked at the reference time.

[0091] In one embodiment, the historical sample behavior sequence (V1, V2…V) is used. t ) and the current sample clicked item (V t+1 Input the student click-through rate prediction model, and the student click-through rate prediction model will output the click rate (V) of the current sample at the next time step (i.e., time t+1). t+1 The first probability of ).

[0092] In one embodiment, the click-through rate (CTR) prediction model is any one of GRU4Rec, BERT4Rec, SASRec, and CL4Srec. GRU4Rec, BERT4Rec, SASRec, and CL4Srec are all CTR prediction models provided by related technologies. Therefore, it can be understood that this application provides a general training architecture, and any CTR prediction model from related technologies can be trained using the method described in this application.

[0093] Step 430: Input the historical sample behavior sequence and the future sample clicked items into the student click rate prediction model to predict the second probability that the future sample clicked items will be clicked after the reference time.

[0094] In one embodiment, the historical sample behavior sequence (V1, V2…V) is used. t ) and future sample click projects (V t+m Input the student click-through rate prediction model, and the student click-through rate prediction model will output the clicked items (V) of future samples after time t+1. t+m The second probability of ).

[0095] Step 440: Train the student click-through rate prediction model based on the error between the probability label of the current sample clicked item and the first probability.

[0096] Optionally, the probability label for the currently clicked item is a hard label. A hard label, also known as a one-hot label, indicates that the proportion of positive samples in the currently clicked item is 1, and the proportion of negative samples is 0. Hard labels are binary classification labels.

[0097] In one embodiment, the click-through rate (CTR) prediction model is trained based on the error between the first probability predicted by the student CTR prediction model and the hard label of the currently clicked item. Illustratively, a loss function L is constructed based on the hard label of the currently clicked item and the first probability. ori According to the loss L ori Train a student click-through rate prediction model.

[0098] Step 450: Obtain the hard tags for future sample clicked items;

[0099] The hard label for a future click item indicates that the percentage of positive samples is 1 and the percentage of negative samples is 0. The hard label is a binary label.

[0100] Step 460: Input the historical sample behavior sequence and the future sample click item into the teacher click rate prediction model to predict the third probability that the future sample click item will be clicked after the reference time; use the third probability as the soft label of the future sample click item.

[0101] Soft labels indicate the proportion of positive samples in future clicked items, denoted as p. p is a decimal between 0 and 1. That is, soft labels are not binary classification labels. Soft labels facilitate learning the deeper knowledge implicit in probability labels. Soft labels achieve noise reduction and purification of information about future clicked items. Soft labels implicitly contain the distribution of positive and negative sample categories. By using the difference between the confidence scores of positive and negative sample categories, the student click-through rate prediction model can better learn and utilize future information.

[0102] The teacher click-through rate (CTR) prediction model is used to guide the training of the student CTR prediction model. The teacher CTR prediction model guides the student CTR prediction model in learning about future clickable items. The parameter size of the teacher CTR prediction model is no smaller than that of the student CTR prediction model.

[0103] In one embodiment, historical sample behavior sequences and future sample click items are input into a teacher click rate prediction model to predict the third probability that a future sample click item will be clicked after time t+1; this third probability is used as a soft label for the future sample click item. The value of the third probability is a decimal between 0 and 1.

[0104] Step 470: Generate probability labels for future sample clicked items based on hard and soft labels;

[0105] In one embodiment, the soft label is multiplied by the softening weight to obtain a third value. The hard label is multiplied by the hardening weight to obtain a fourth value; the sum of the softening and hardening weights is one. The third and fourth values ​​are summed to obtain the probability label for future sample clicks.

[0106] To illustrate, the following formula is used:

[0107]

[0108] in, The label representing the probability of clicking on an item in the future sample, v t+m Indicates the future sample click item, s t P represents the sequence of click behavior in historical samples. T (v t+m |s t ) represents the soft label (i.e., the third probability) output by the teacher click rate prediction model, y represents the hard label of the future sample click item, α represents the softening weight, and (1-α) represents the hardening weight.

[0109] In one embodiment, the softening weight of a future sample click item is determined from multiple candidate softening weights. Illustratively, a first softening weight matching the soft label is selected from the multiple candidate softening weights; this first softening weight is used as the softening weight of the soft label. Illustratively, multiple candidate softening weights, such as 0.2, 0.5, 0.7, 0.9, etc., are preset. Based on the calculated soft label value, a softening weight for performing the soft label softening operation is selected from 0.2, 0.5, 0.7, and 0.9. Illustratively, a soft label value of 0.9 indicates that the future sample click item is relatively clean, so a larger softening weight of 0.9 is selected, with a corresponding hardening weight of 0.1. Illustratively, a soft label value of 0.6 indicates that the future sample click item has a lot of noise, so a smaller softening weight of 0.2 is selected, with a corresponding hardening weight of 0.8.

[0110] Step 480: Train the student click-through rate prediction model based on the error between the probability label of the future sample clicked item and the second probability.

[0111] In one embodiment, the cross-entropy loss function value is calculated based on the probability label and second probability of the item clicked in the future sample; the click-through rate prediction model is trained based on the cross-entropy loss function value.

[0112] Illustratively, the augmentation loss is constructed based on the probability label and second probability of future sample clicks on items. In one embodiment, the augmentation loss is represented using cross-entropy. The formula for the augmentation loss is as follows:

[0113]

[0114] Where u represents the user, n is n in the total length (t+n) of the user sample behavior sequence, t is the time t, and v t+m For future sample click projects, s represents the probability label of a future sample clicking an item. t P(v) represents the historical sample click behavior sequence. t+m |s t ) represents the second probability output by the student click-through rate prediction model. It means (1-P(v) t+m |s t The value of ).

[0115] It is understandable that the augmentation loss is a loss constructed across three dimensions: multiple users, multiple current sample click items, and multiple future sample click items. In this embodiment, a detailed analysis has been conducted specifically on one current sample click item and one future sample click item within a user sample behavior sequence.

[0116] In summary, the above embodiments train a click-through rate (CTR) prediction model using a knowledge distillation architecture, which includes a student model (student CTR prediction model) and a teacher model (teacher CTR prediction model). The teacher model generates soft tags for future clicked items, and the student model is trained based on these soft and hard tags. The knowledge distillation architecture facilitates training the student model on top of the teacher model, thus improving the training efficiency of the student model.

[0117] Furthermore, soft labels enable the denoising and purification of label information for clicked items in future samples. Soft labels implicitly contain the distribution of positive and negative sample categories. By using the difference between the confidence scores of positive and negative sample categories, it is beneficial for the teacher model to guide the student model during training. The knowledge guided by the teacher model is more reliable and can improve the prediction accuracy of the trained student model.

[0118] Figure 5 A schematic diagram illustrating an exemplary embodiment of this application is shown. Figure 5 The training architecture 500 for the click-through rate prediction model is shown. Figure 5 A knowledge distillation method is used to train the click-through rate prediction model. Figure 5 The model 501 (i.e., the student model) and the model 502 (i.e., the teacher model) for predicting the student click-through rate in the current training round are shown in the knowledge distillation. Figure 5 The training architecture aims to train the student click-through rate prediction model 501 for the current training round.

[0119] Figure 5 This illustrates the acquisition of user sample behavior sequences (V1, V2, ..., V...). t ...V t+1 V t+m V t+n Obtain historical sample behavior sequences (V1, V2, ..., V) from user sample behavior sequences. t 503, Current sample clicked item 504 (V) t+1 ) and future sample click item 505 (V t+m The historical sample behavior sequence 503 and the current sample clicked item 504 are input into the student click-through rate prediction model 501 for the current training round. The student click-through rate prediction model 501 for the current training round will output the predicted first probability 506. Based on the first probability 506 and the hard label 507 of the current sample clicked item, the loss is constructed to train the student click-through rate prediction model 501 for the current training round.

[0120] Input the future sample click item 505 and the historical sample behavior sequence 503 into the student click-through rate prediction model 501 in the current training round. The student click-through rate prediction model 501 in the current training round will output the predicted second probability 508. Based on the second probability 508, the soft label 509 of the future sample click item, and the hard label 510 of the future sample click item, construct the loss to train the student click-through rate prediction model 501 in the current training round.

[0121] Figure 5 It also shows how to obtain the soft label 509 of future sample click items. Figure 5 The model shows how a teacher click rate prediction model 502, which inputs future sample click items 505 and historical sample behavior sequences 503 into the current training round, outputs soft labels 509 for future sample click items.

[0122] Figure 5 It also shows that the teacher click-through rate prediction model 502 for the current training round is obtained based on the teacher click-through rate prediction model 511 from the previous training round and the student click-through rate prediction model 501 for the current training round.

[0123] Figure 6 A flowchart illustrating a training method for a click-through rate prediction model provided in an exemplary embodiment of this application is shown, wherein the method is... Figure 1 The computer device shown is used to illustrate, for example, the method including:

[0124] Step 610: Obtain historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences;

[0125] User sample behavior sequence: refers to the click behavior sequence of a single user in the training dataset. The user sample behavior sequence includes multiple sample click items arranged in chronological order. For example, the user sample behavior sequence includes sample click items V1 at time t, V2 at time t, ..., and Vn at time t+n, arranged in chronological order. t+n Clickable items refer to items on the terminal interface that users can click. Examples include advertisements, news items, headlines, events, and entity words.

[0126] Historical sample behavior sequence: This includes multiple sample click items in the user sample behavior sequence that are prior to a reference time. Optionally, the reference time is time t+1. Optionally, the historical sample behavior sequence includes t sample click items in the user sample behavior sequence that are prior to time t+1 (i.e., t sample click items corresponding to the previous t times). For example, the historical sample behavior sequence includes sample click items V1 at the first time, V2 at the second time, ..., and V at time t, arranged in sequence. t t is a positive integer greater than 1.

[0127] Current sample click item: refers to the sample click item in the user sample behavior sequence corresponding to the reference time. Optionally, the reference time is time t+1, which can also be called the current time. The current sample click item can also be represented as V. t+1 .

[0128] Future sample click items: These refer to sample click items in the user sample behavior sequence that occur after a reference time. Optionally, the reference time is time t+1. That is, future sample click items are sample click items that occur after the current sample click item. For example, future sample click items can be represented as a user sample behavior sequence (V1, V2, ... V...). t+1 V t+m V t+n V in ) t+m Where m is greater than 1 and m is less than n. In one embodiment, from the user sample behavior sequence (V1, V2, ... V... t+1 V t+m V t+n The item V in the current sample is located in the middle. t+1 In subsequent sample click projects, future sample click projects V are randomly selected. t+m .

[0129] Step 620: Input the historical sample behavior sequence and the current sample clicked item into the student click rate prediction model of the current training round to predict the first probability that the current sample clicked item will be clicked at the reference time.

[0130] In one embodiment, the historical sample behavior sequence (V1, V2…V) is used. t ) and the current sample clicked item (V t+1 Input the click-through rate (CTR) prediction model, and the CTR prediction model will output the click value (V) of the current sample at the next time step (i.e., time t+1). t+1 The first probability of ).

[0131] In one embodiment, the click-through rate (CTR) prediction model is any one of GRU4Rec, BERT4Rec, SASRec, and CL4Srec. GRU4Rec, BERT4Rec, SASRec, and CL4Srec are all CTR prediction models provided by related technologies. Therefore, it can be understood that this application provides a general training architecture, and any CTR prediction model from related technologies can be trained using the method described in this application.

[0132] Step 630: Input the historical sample behavior sequence and the future sample click item into the student click rate prediction model of the current training round to predict the second probability that the future sample click item will be clicked after the reference time.

[0133] In one embodiment, the historical sample behavior sequence (V1, V2…V) is used. t ) and future sample click projects (V t+m Input the click-through rate (CTR) prediction model for the current training round, and the CTR prediction model will output the clicked item (V) of the future sample at the next time step (i.e., time t+1). t+m The second probability of ).

[0134] Step 640: Based on the error between the probability label of the clicked item in the current sample and the first probability, train the student click rate prediction model for the current training round;

[0135] Optionally, the probability label of the currently clicked item is a hard label. A hard label indicates that the proportion of positive samples is 1 and the proportion of negative samples is 0. The hard label is a binary classification label. Illustratively, a loss L is constructed based on the hard label of the currently clicked item and the first probability. ori According to the loss L ori Train the student click-through rate prediction model for the current training round.

[0136] Step 650: Obtain the hard tags for future sample clicked items;

[0137] The hard label for a future click item indicates that the percentage of positive samples is 1 and the percentage of negative samples is 0. The hard label is a binary label.

[0138] Step 660: Generate the teacher click-through rate prediction model for the current training round based on the teacher click-through rate prediction model from the previous training round and the student click-through rate prediction model for the current training round.

[0139] In one embodiment, the model parameters of the teacher click-through rate prediction model from the previous training round are multiplied by a first decay weight to obtain a first value. The model parameters of the student click-through rate prediction model from the current training round are multiplied by a second decay weight to obtain a second value; the sum of the first and second decay weights is one. The first and second values ​​are summed to obtain the model parameters of the teacher click-through rate prediction model for the current training round.

[0140] To illustrate, the formula is as follows:

[0141] θ T (t)=dθ T (t-1)+(1-d)θ(t);

[0142] Where, θ T (t-1) represents the model parameters of the teacher click-through rate prediction model in the previous training round, and θ(t) represents the model parameters of the student click-through rate prediction model in the current training round. T (t) represents the model parameters of the teacher click-through rate prediction model in the current training round, where d represents the first decay weight and (1-d) represents the second decay weight. The decay weights characterize the parameter decay of the teacher prediction model in the current training round compared to the teacher prediction model in the previous training round.

[0143] It is understandable that the above formula indicates that the teacher click-through rate prediction model is generated using the EMA (Exponential Moving Average) method.

[0144] It is understandable that the generation method of the above teacher click-through rate prediction model indicates that knowledge distillation will be performed on future sample click items using a self-distillation method. The self-distillation method is a knowledge distillation method where the student model and the teacher model are the same model.

[0145] Step 670: Input the historical sample behavior sequence and the future sample click item into the teacher click rate prediction model of the current training round to predict the third probability that the future sample click item will be clicked at the reference time; use the third probability as the soft label of the future sample click item.

[0146] Soft labels indicate the proportion of positive samples in future clicked items, denoted as p. p is a decimal between 0 and 1. That is, soft labels are not binary classification labels. Soft labels facilitate learning the deeper knowledge implicit in probability labels. Soft labels achieve noise reduction and purification of information about future clicked items. Soft labels implicitly contain the distribution of positive and negative sample categories. By using the difference between the confidence scores of positive and negative sample categories, the student click-through rate prediction model can better learn and utilize future information.

[0147] In one embodiment, under the current training round, the historical sample behavior sequence and the future sample click item are input into the teacher click rate prediction model of the current training round to predict the third probability of the future sample click item being clicked; the third probability is used as a soft label for the future sample click item. The value of the third probability is a decimal between 0 and 1.

[0148] Step 680: Generate probability labels for future sample clicked items based on hard and soft labels;

[0149] In one embodiment, at the current training epoch, the soft label is multiplied by the softening weight to obtain a third value. The hard label is multiplied by the hardening weight to obtain a fourth value; the sum of the softening and hardening weights is one. The third and fourth values ​​are summed to obtain the probability label for future sample clicks.

[0150] To illustrate, the following formula is used:

[0151]

[0152] in, The label representing the probability of clicking on an item in the future sample, v t+m Indicates the future sample click item, s t P represents the sequence of click behavior in historical samples. T (v t+m |s t ) represents the soft label (i.e., the third probability) output by the teacher click rate prediction model, y represents the hard label of the future sample click item, α represents the softening weight, and (1-α) represents the hardening weight.

[0153] Step 690: Train the student click-through rate prediction model for the current training round based on the error between the probability label of the future sample clicked item and the second probability.

[0154] In one embodiment, under the current training round, the cross-entropy loss function value is calculated based on the probability label and second probability of the clicked item in the future sample; the click-through rate prediction model is trained based on the cross-entropy loss function value.

[0155] Illustratively, the augmentation loss is constructed based on the probability label and second probability of future sample clicks on items. In one embodiment, the augmentation loss is represented using cross-entropy. Illustratively, the formula for the augmentation loss is as follows:

[0156]

[0157] Where u represents the user, n is n in the total length (t+n) of the user sample behavior sequence, t is the time t, and v t+m For future sample click projects, s represents the probability label of a future sample clicking an item. t P(v) represents the historical sample click behavior sequence. t+m |s t ) represents the second probability output by the student prediction model. It means (1-P(v) t+m |s t The value of ).

[0158] It is understandable that the augmentation loss is a loss constructed across three dimensions: multiple users, multiple current sample click items, and multiple future sample click items. In this embodiment, a detailed analysis has been conducted specifically on one current sample click item and one future sample click item within a user sample behavior sequence.

[0159] In summary, the above embodiments provide a method for generating a teacher model (teacher click-through rate prediction model). The teacher model is essentially derived from student models in previous training rounds and the student model in the current training round. Therefore, the knowledge distillation method in the above embodiments is self-distillation. The self-distillation method used in the above embodiments to generate soft labels can improve the label quality of future sample click items.

[0160] Figure 7 This illustration shows a schematic diagram of an exemplary embodiment of the present application. Figure 7 The training architecture 700 of the click-through rate prediction model is shown. Figure 7 A knowledge distillation method was used to train the student click-through rate prediction model. Figure 7 The student click-through rate prediction model 701 (i.e., the student model) and the teacher click-through rate prediction model 702 (i.e., the teacher model) in knowledge distillation are shown. Figure 7 The purpose of the training architecture is to train the student click-through rate prediction model 701 for the current training round.

[0161] Figure 7 This illustrates the acquisition of user sample behavior sequences (V1, V2, ..., V...). t ...V t+1 V t+m V t+nObtain historical sample behavior sequences (V1, V2, ..., V) from user sample behavior sequences. t 703, Current sample clicked item 704 (V) t+1 ) and future sample click project 705 (V t+m The historical sample behavior sequence 703 and the current sample clicked item 704 are input into the student click-through rate prediction model 701. The student click-through rate prediction model 701 will output the predicted first probability (…). Figure 7 (Not shown). A loss L is constructed based on the first probability and the hard label of the currently clicked item. ori Train the student click-through rate prediction model 701.

[0162] Input the future sample click item 705 and the historical sample behavior sequence 703 into the student click-through rate prediction model 701. The student click-through rate prediction model 701 will output the predicted second probability ( Figure 7 (Not shown). Based on the second probability, the soft label of the future sample click item, and the hard label of the future sample click item, construct the loss L. aug Train the student click-through rate prediction model 701 for the current training round.

[0163] Figure 7 It also shows how to obtain soft labels for future sample click items. Figure 7 The diagram shows how inputting future sample click items 705 and historical sample behavior sequences 703 into the teacher click rate prediction model 702 outputs soft labels for future sample click items. Figure 7 The method shown for obtaining soft tags is self-distillation.

[0164] Figure 7 The generation method of teacher click-through rate prediction model 702 is also shown. The parameters of the teacher click-through rate prediction model for the current training round are obtained by applying an exponential moving average strategy to the parameters (θ1, θ2, ..., θk) of the student click-through rate prediction model from previous training rounds and the parameters of the student click-through rate prediction model for the current training round. The parameters of student click-through rate prediction model 701 are continuously stored and updated to facilitate the generation of teacher click-through rate prediction model 702 for the current training round.

[0165] Understandable, Figure 7 The training architecture shown is 700, and the student click-through rate prediction model uses the user's behavior sequence s at t+1. t ={v1、…、v t As input, it outputs a user's expression vector, which is then used to predict the currently clicked sample item (item) V. t+1 The student click-through rate prediction model expands the original positive value V from multiple items that are expected to be clicked in the future. t+1This achieves future information gain. To mitigate additional noise, Figure 7 The training architecture uses a self-distillation method with EMA to generate soft labels for these positive boosts, and then combines these soft labels with their hard labels for training.

[0166] Figure 7 The items to be clicked in the future are natural, high-quality augmented samples of the currently clicked items. Optionally, items to be augmented can be selected using an equal-probability method. Optionally, items to be clicked in the future can be randomly sampled with equal probability 1 / (nt) (1≤m≤nt).

[0167] To avoid additional noise from future enhancements, Figure 7 Self-distillation is used to reweight the newly enhanced positive sample V. t+m The prediction is made by relying on self-distillation to provide more reliable and stable soft labels for these samples. Optionally, the teacher click-through rate (CTR) prediction model can be obtained through two methods. First, the best-performing student CTR prediction model on the validation set of past training epochs can be directly used as the teacher CTR prediction model for the current training epoch. Second, the teacher CTR prediction model for the current training epoch can be obtained by performing EMA on the student CTR prediction model from past training epochs. Formally, the parameter θ of the teacher CTR prediction model at time t... T (t) is derived from the parameters θ of the previous teacher click-through rate prediction model. T (t-1) and the parameter θ(t) of the current student click-through rate prediction model are generated, with a decay rate of d, and the specific formula is as follows:

[0168] θ T (t)=dθ T (t-1)+(1-d)θ(t);

[0169] Next, the predicted probability P of the teacher (i.e., the soft label) will be... T (v t+m |s t The labels are combined with hard labels y with weight α to obtain the final labels of the augmented samples. for: y = 1 represents the positive sample V t+m y = 0 represents a random negative sample. α is the softening weight. Finally, the augmenting loss L... aug Formalized as:

[0170]

[0171] By using a set of historical models to guide the learning of augmented positive samples, the risk of involving additional noise can be reduced and the label quality can be improved.

[0172] Table 1 below shows that, based on the GRU4Rec, BERT4Rec, SASRec, and CL4SRec model structures, plus the FASRec algorithm (i.e., the algorithm provided in this application), significant improvements can be achieved on three public datasets (Yelp, Beauty, and Sports) through rigorous experiments. Furthermore, improvements are also achieved in online A / B tests (a significance testing scheme). In Table 1, HR@1, HR@5, HR@10, HR@20, NDCG@5, NDCG@10, and NDCG@20 are evaluation metrics. HR (HitRatio)@K is one evaluation metric. NDCG (Normalized Discounted Cumulative Gain)@K is another evaluation metric.

[0173] Table 1

[0174]

[0175]

[0176] In experiments, FASRec (the algorithm provided in this application) achieved significant and consistent improvements in offline and online evaluations using different base sequence recommendation models, demonstrating the effectiveness and universality of FASRec.

[0177] Figure 8 This application illustrates a training apparatus for a click-through rate prediction model provided in an exemplary embodiment. The apparatus includes:

[0178] The acquisition module 801 is used to acquire historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences. The user sample behavior sequence includes multiple sample click items arranged in chronological order, the historical sample behavior sequence includes multiple sample click items located before the reference time, the current sample click item is the sample click item corresponding to the reference time, and the future sample click item is the sample click item located after the reference time; t is a positive integer greater than or equal to 1.

[0179] Prediction module 802 is used to input the historical sample behavior sequence and the current sample clicked item into the click rate prediction model to predict the first probability that the current sample clicked item will be clicked at a reference time.

[0180] The prediction module 802 is also used to input the historical sample behavior sequence and the future sample click item into the click rate prediction model to predict the second probability that the future sample click item will be clicked after the reference time.

[0181] Training module 803 is used to train a click-through rate prediction model based on the error between the probability label of the clicked item in the current sample and the first probability; and to train a click-through rate prediction model based on the error between the probability label of the clicked item in future samples and the second probability.

[0182] In an optional embodiment, the training module 803 is further configured to obtain hard labels and soft labels for future sample click items, wherein the hard label indicates that the proportion of positive sample categories of future sample click items is 1, and the soft label indicates that the proportion of positive sample categories of future sample click items is p, where p is greater than or equal to 0 and p is less than or equal to 1; generate probability labels for future sample click items based on the hard labels and soft labels; and train a click-through rate prediction model based on the error between the probability labels and the second probability of future sample click items.

[0183] In an optional embodiment, the click-through rate prediction model is a student click-through rate prediction model. The training module 803 is further configured to input historical sample behavior sequences and future sample click items into the teacher click-through rate prediction model to predict the third probability that a future sample click item will be clicked after a reference time; the parameter size of the teacher click-through rate prediction model is not less than the parameter size of the student click-through rate prediction model; and the third probability is used as a soft label for the future sample click item.

[0184] In an optional embodiment, the student click-through rate (CTR) prediction model is the student CTR prediction model for the current training round, and the teacher CTR prediction model is the teacher CTR prediction model for the current training round. The apparatus also includes a generation module 804. The generation module 804 is configured to generate a teacher CTR prediction model for the current training round based on the teacher CTR prediction model from the previous training round and the student CTR prediction model for the current training round.

[0185] In an optional embodiment, the generation module 804 is further configured to multiply the model parameters of the teacher click-through rate prediction model of the previous training round by a first decay weight to obtain a first value; multiply the model parameters of the student click-through rate prediction model of the current training round by a second decay weight to obtain a second value; the sum of the first decay weight and the second decay weight is one; and sum the first value and the second value to obtain the model parameters of the teacher click-through rate prediction model of the current training round.

[0186] In an optional embodiment, the training module 803 is further configured to multiply the soft label by the softening weight to obtain a third value; multiply the hard label by the hardening weight to obtain a fourth value; the sum of the softening weight and the hardening weight is one; and sum the third value and the fourth value to obtain the probability label of the future sample click item.

[0187] In an optional embodiment, the training module 803 is further configured to select a first softening weight that matches the soft label from a plurality of candidate softening weights; and use the first softening weight as the softening weight of the soft label.

[0188] In an optional embodiment, the acquisition module 801 is further configured to randomly extract future sample click items from the sample click items located after the current sample click item in the user sample behavior sequence.

[0189] In an optional embodiment, the training module 803 is further configured to calculate the cross-entropy loss function value based on the probability label and second probability of the clicked item in the future sample; and to train the click-through rate prediction model based on the cross-entropy loss function value.

[0190] In summary, by acquiring historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences, a click-through rate (CTR) prediction model is trained based on the historical sample behavior sequences and current sample click items; a CTR prediction model is also trained based on the historical sample behavior sequences and future sample click items.

[0191] Compared to related technologies, this application not only performs the task of predicting the clicked items of the current sample, but also the task of predicting the clicked items of future samples. Historical sample behavior sequences implicitly contain user preference features. This application takes into account that user preference features are related not only to the clicked items of the current sample, but also to the clicked items of future samples.

[0192] In this application, future sample click items will be used as enhanced positive samples for training, thus improving the training data. This data augmentation of the training samples can alleviate the sparsity problem of user click items in real-world usage, as well as the variability and irregularity of user click behavior.

[0193] Figure 9 This is a schematic diagram illustrating the structure of a computer device according to an exemplary embodiment. The computer device 900 includes a Central Processing Unit (CPU) 901, a system memory 904 including Random Access Memory (RAM) 902 and Read-Only Memory (ROM) 903, and a system bus 905 connecting the system memory 904 and the CPU 901. The computer device 900 also includes a basic input / output system (I / O system) 906 that facilitates information transfer between various components within the computer device, and a mass storage device 907 for storing an operating system 913, application programs 914, and other program modules 915.

[0194] The basic input / output system 906 includes a display 908 for displaying information and an input device 909 for user input, such as a mouse or keyboard. Both the display 908 and the input device 909 are connected to the central processing unit 901 via an input / output controller 910 connected to the system bus 905. The basic input / output system 906 may also include the input / output controller 910 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 910 also provides output to a display screen, printer, or other types of output devices.

[0195] The mass storage device 907 is connected to the central processing unit 901 via a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer device-readable media provide non-volatile storage for the computer device 900. That is, the mass storage device 907 may include computer device-readable media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0196] Without loss of generality, the computer device readable medium may include computer device storage media and communication media. Computer device storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer device readable instructions, data structures, program modules, or other data. Computer device storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM, digital video disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer device storage media are not limited to the above-mentioned types. The system memory 904 and mass storage device 907 described above can be collectively referred to as memory.

[0197] According to various embodiments of this disclosure, the computer device 900 can also be connected to a remote computer device on a network, such as the Internet. That is, the computer device 900 can be connected to a network 911 via a network interface unit 912 connected to the system bus 905, or it can use the network interface unit 912 to connect to other types of networks or remote computer device systems (not shown).

[0198] The memory also includes one or more programs stored in the memory, and the central processing unit 901 executes the one or more programs to implement all or part of the steps of the above-mentioned click-through rate prediction model training method.

[0199] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the training method for the click-through rate prediction model provided in the above method embodiments.

[0200] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the training method for the click-through rate prediction model provided in the above-described method embodiments.

[0201] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0202] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0203] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A training method for a click-through rate prediction model, characterized in that, The method includes: Obtain historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences; the user sample behavior sequences include multiple sample click items arranged in chronological order, the historical sample behavior sequences include multiple sample click items located before a reference time, the current sample click items are sample click items corresponding to the reference time, and the future sample click items are sample click items located after the reference time. The historical sample behavior sequence and the current sample clicked item are input into the click rate prediction model to predict the first probability that the current sample clicked item is clicked at the reference time. The historical sample behavior sequence and the future sample click item are input into the click rate prediction model to predict the second probability that the future sample click item will be clicked after the reference time. The click-through rate prediction model is trained based on the error between the probability label of the current sample clicked item and the first probability; and, a hard label for the future sample clicked item is obtained, and the historical sample behavior sequence and the future sample clicked item are input into the teacher click-through rate prediction model to predict the third probability that the future sample clicked item will be clicked after a reference time. The third probability is used as the soft label for the future sample clicked item. Based on the hard label and the soft label, a probability label for the future sample clicked item is generated. The click-through rate prediction model is trained based on the error between the probability label of the future sample clicked item and the second probability.

2. The method according to claim 1, characterized in that, The hard label indicates that the proportion of positive sample categories in the future sample click item is 1, and the soft label indicates that the proportion of positive sample categories in the future sample click item is p, where p is greater than or equal to 0 and p is less than or equal to 1.

3. The method according to claim 2, characterized in that, The click-through rate prediction model is a student click-through rate prediction model; the parameter size of the teacher click-through rate prediction model is not less than the parameter size of the student click-through rate prediction model.

4. The method according to claim 3, characterized in that, The student click-through rate prediction model is the student click-through rate prediction model for the current training round, and the teacher click-through rate prediction model is the teacher click-through rate prediction model for the current training round; the method further includes: Based on the teacher click-through rate prediction model from the previous training round and the student click-through rate prediction model from the current training round, the teacher click-through rate prediction model for the current training round is generated.

5. The method according to claim 4, characterized in that, The step of generating the teacher click-through rate prediction model for the current training round based on the teacher click-through rate prediction model from the previous training round and the student click-through rate prediction model for the current training round includes: The model parameters of the teacher click-through rate prediction model from the previous training round are multiplied by the first decay weight to obtain the first value; The model parameters of the student click-through rate prediction model in the current training round are multiplied by the second decay weight to obtain the second value; the sum of the first decay weight and the second decay weight is one. The first value and the second value are summed to obtain the model parameters of the teacher click-through rate prediction model for the current training round.

6. The method according to claim 2, characterized in that, The step of generating probability labels for future sample click items based on the hard labels and the soft labels includes: Multiply the soft label by the softening weight to obtain the third value; The hard label is multiplied by the hardening weight to obtain the fourth value; the sum of the softening weight and the hardening weight is one. The probability label of the future sample click item is obtained by summing the third value and the fourth value.

7. The method according to claim 6, characterized in that, The method further includes: From multiple candidate softening weights, select the first softening weight that matches the soft label; use the first softening weight as the softening weight of the soft label.

8. The method according to any one of claims 1 to 7, characterized in that, The process of obtaining the future sample click item includes: The future sample click item is randomly selected from the sample click items that follow the current sample click item in the user sample behavior sequence.

9. The method according to any one of claims 1 to 7, characterized in that, The step of training the click-through rate prediction model based on the error between the probability label of the future sample clicked item and the second probability includes: Based on the probability labels of the future sample clicked items and the second probability, the cross-entropy loss function value is calculated; based on the cross-entropy loss function value, the click-through rate prediction model is trained.

10. A training device for a click-through rate prediction model, characterized in that, The device includes: The acquisition module is used to acquire historical sample behavior sequences, current sample click items, and future sample click items from user sample behavior sequences; the user sample behavior sequences include multiple sample click items arranged in chronological order, the historical sample behavior sequences include multiple sample click items located before a reference time, the current sample click items are sample click items corresponding to the reference time, and the future sample click items are sample click items located after the reference time. The prediction module is used to input the historical sample behavior sequence and the current sample click item into the click rate prediction model to predict the first probability that the current sample click item is clicked at the reference time; The prediction module is further configured to input the historical sample behavior sequence and the future sample click item into the click rate prediction model to predict a second probability that the future sample click item will be clicked after the reference time. The training module is used to train the click-through rate prediction model based on the error between the probability label of the current sample clicked item and the first probability; and to obtain the hard label of the future sample clicked item, and to input the historical sample behavior sequence and the future sample clicked item into the teacher click-through rate prediction model to predict the third probability that the future sample clicked item will be clicked after a reference time, and to use the third probability as the soft label of the future sample clicked item, to generate the probability label of the future sample clicked item based on the hard label and the soft label, and to train the click-through rate prediction model based on the error between the probability label of the future sample clicked item and the second probability.

11. The apparatus according to claim 10, characterized in that, The hard label indicates that the proportion of positive sample categories in the future sample click item is 1, and the soft label indicates that the proportion of positive sample categories in the future sample click item is p, where p is greater than or equal to 0 and p is less than or equal to 1.

12. The apparatus according to claim 11, characterized in that, The click-through rate prediction model is a student click-through rate prediction model. The parameter size of the teacher click-through rate prediction model is not less than the parameter size of the student click-through rate prediction model.

13. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the training method for the click-through rate prediction model as described in any one of claims 1 to 9.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is loaded and executed by a processor to implement the training method for the click-through rate prediction model as described in any one of claims 1 to 9.

15. A computer program product, characterized in that, The computer program product stores a computer program that is loaded and executed by a processor to implement the training method for the click-through rate prediction model as described in any one of claims 1 to 9.

Images