A training sample acquisition method, conversion rate estimation model generation method and device

Abstract

The present application discloses a training sample acquisition method, a conversion rate estimation model generation method and an apparatus, wherein the method comprises: after obtaining a first training sample set, inputting the negative samples and time features in the first training sample set into a scattering model. The time features are determined based on the reflow time of the positive samples in the first training sample set, and the scattering model is pre-trained and used to determine the temporal distribution of negative samples, and the temporal distribution of negative samples is determined based on the negative samples and the time features. After determining the temporal distribution of the negative samples, the second training sample set is determined based on the time distribution, so that the ratio of positive samples to negative samples in the determined second training sample set meets the preset conditions. Since the present application refers to the reflow time of the positive samples when determining the temporal distribution of the negative samples, the distribution of the negative samples can be accurately scattering, thereby improving the stability of the distribution ratio of the positive and negative samples.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for acquiring training samples, and a method and device for generating a conversion rate prediction model. Background Art

[0002] Currently, streaming update technology is used to update the post-click conversion rate estimation (CVR) model for precise ranking. This allows the CVR model to quickly capture changes in behavioral characteristics, thereby improving the accuracy of ad push. Streaming updates rely heavily on the stability of the data stream, particularly the stability of the positive-negative sample ratio, which in turn depends primarily on the distribution of negative samples.

[0003] However, due to the limitations of the fragmentation model itself, there are large differences when determining the distribution of negative samples, which makes the distribution of the positive and negative sample ratios unstable and affects the accuracy of the estimation. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a training sample acquisition method, a conversion rate estimation model generation method and device to improve the stability of the positive and negative sample ratio distribution, thereby improving the estimation accuracy.

[0005] To achieve the above objectives, the technical solutions provided by this application are as follows:

[0006] In a first aspect of the present application, a method for obtaining a training sample is provided, the method comprising:

[0007] Obtain a first training sample set, the first training sample set including positive samples and negative samples, the positive samples are samples in which conversion behavior occurs after being clicked, and the negative samples are samples in which no conversion behavior occurs after being clicked;

[0008] Inputting the negative sample and the time feature into a scattering model, obtaining a time distribution corresponding to the negative sample output by the scattering model, wherein the time feature is determined based on the reflow time of the positive sample, and the scattering model is pre-trained to determine the time distribution of the negative sample;

[0009] A second training sample set is determined according to the time distribution, and a ratio of positive samples to negative samples in the second training sample set meets a preset condition.

[0010] In a second aspect of the present application, a method for generating a conversion rate prediction model is provided, the method comprising:

[0011] Obtaining a training sample set, where the training sample set is obtained based on the method of the first aspect, wherein the ratio of positive samples to negative samples in the training sample set meets a preset condition, the positive samples are samples in which conversion behavior occurs after being clicked, and the negative samples are samples in which no conversion behavior occurs after being clicked;

[0012] The initial model is trained using the training sample set to generate a conversion rate estimation model.

[0013] In a third aspect of the present application, a training sample acquisition device is provided, the device comprising:

[0014] A first acquisition unit is configured to acquire a first training sample set, wherein the first training sample set includes positive samples and negative samples, wherein the positive samples are samples in which conversion occurs after being clicked, and the negative samples are samples in which no conversion occurs after being clicked;

[0015] a second acquisition unit, configured to input the negative sample and a time feature into a scattering model, and obtain a time distribution corresponding to the negative sample output by the scattering model, wherein the time feature is determined based on the reflow time of the positive sample, and the scattering model is pre-trained and configured to determine the time distribution of the negative sample;

[0016] A determining unit is configured to determine a second training sample set according to the time distribution, wherein a ratio of positive samples to negative samples in the second training sample set meets a preset condition.

[0017] In a fourth aspect of the present application, a device for generating a conversion rate prediction model is provided, the device comprising:

[0018] an acquisition unit, configured to acquire a training sample set, wherein the training sample set is acquired based on the method of the first aspect, wherein the ratio of positive samples to negative samples in the training sample set meets a preset condition, wherein the positive samples are samples in which conversion behavior occurs after being clicked, and the negative samples are samples in which no conversion behavior occurs after being clicked;

[0019] The generating unit is used to train the initial model using the training sample set to generate a conversion rate estimation model.

[0020] In a fifth aspect of the present application, an electronic device is provided, the device comprising: a processor and a memory;

[0021] The memory is used to store instructions or computer programs;

[0022] The processor is configured to execute the instructions or computer program in the memory so that the electronic device executes the method described in the first aspect.

[0023] In a sixth aspect of the present application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a device, the device executes the method described in the first aspect.

[0024] In a seventh aspect of the present application, a computer program product is provided, wherein the computer program product comprises a computer program / instructions, and when the computer program / instructions are executed by a processor, the method described in the first aspect is implemented.

[0025] It can be seen that the embodiments of the present application have the following beneficial effects:

[0026] In an embodiment of the present application, after obtaining the first training sample set, the negative samples and time features in the first training sample set are input into the scattering model. Among them, the time feature is determined based on the reflow time of the positive samples in the first training sample set, and the scattering model is pre-trained and used to determine the temporal distribution of the negative samples. The time distribution of the negative samples will be determined based on the negative samples and the time features. After determining the time distribution of the negative samples, the second training sample set will be determined based on the time distribution, so that the ratio of positive samples and negative samples in the second training sample set determined meets the preset conditions. Since the present application refers to the reflow time of the positive samples when determining the time distribution of the negative samples, the distribution of the negative samples can be accurately scattering, thereby improving the stability of the distribution ratio of positive and negative samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 A flow chart of a method for obtaining training samples provided in an embodiment of the present application;

[0029] Figure 2a A schematic diagram of a broken-up model structure provided in an embodiment of the present application;

[0030] Figure 2b A schematic diagram of a time feature determination scenario provided in an embodiment of the present application;

[0031] Figure 3 A flow chart of a method for generating a conversion rate prediction model provided in an embodiment of the present application;

[0032] Figure 4 A schematic diagram of the structure of a training sample acquisition device provided in an embodiment of the present application;

[0033] Figure 5 A flow chart of a method for generating a conversion rate prediction model provided in an embodiment of the present application;

[0034] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0036] In advertising application scenarios, clicks generally occur within a very short time after ad exposure, while conversions after clicks usually have a longer delay, perhaps a few minutes, a few days, or even longer. This results in the training data used in the conversion rate CVR estimation model being biased data with incomplete reflux, resulting in a large deviation in the distribution ratio of positive and negative samples, which in turn causes estimation deviation. And when the CVR model is updated in real time, the estimation deviation will be more serious because there are more biased samples in the real-time data. Furthermore, since different ads correspond to different conversion behaviors, the reflux time corresponding to different conversion behaviors may also be different. For example, generally speaking, shallow conversions (such as application installation, activation, registration, etc.) can basically return data within 1 day, while deep conversions (such as purchases) basically require more than a week to return data.

[0037] In order to improve the prediction accuracy of the conversion rate prediction model, the conversion rate prediction model will be trained using positive and negative samples with a stable distribution. To obtain positive and negative samples with a stable distribution, the present application provides a training sample acquisition method. First, a first training sample set is obtained. The first training sample set includes positive samples and negative samples. The distribution ratio of positive and negative samples in the first training sample set may not meet the preset conditions. Then, the negative samples and time features are input into a pre-trained scattering model to use the scattering model to determine the time distribution corresponding to the negative samples, and a second training sample set is determined based on the time distribution. The distribution ratio of positive and negative samples in the second training sample set meets the preset conditions, thereby providing positive and negative samples with a stable distribution for training the conversion rate prediction model and improving the prediction accuracy of the conversion rate prediction model.

[0038] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0039] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operation of the disclosed technical solution based on the prompt message.

[0040] As an optional but non-limiting implementation, in response to a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0041] It is understandable that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0042] To facilitate understanding of the technical solutions provided in the embodiments of the present application, they will be described below with reference to the accompanying drawings.

[0043] See also Figure 1 , which is a flow chart of a training sample acquisition method provided by an embodiment of the present application. The method can be executed by an acquisition device, which can be an electronic device or a server. Among them, the electronic device can include a mobile phone, a tablet computer, a laptop computer, a vehicle terminal, a wearable electronic device, an all-in-one machine, a smart home device and other devices with communication functions, and can also be a virtual machine or a device simulated by a simulator. Figure 1 As shown, the method may include the following steps:

[0044] S101: Acquire a first training sample set, where the first training sample set includes positive samples and negative samples.

[0045] In this embodiment, to train the CVR prediction model, a first training sample set is obtained. The distribution ratio of positive and negative samples included in this first training sample set may not meet the preset conditions. The specific performance of positive and negative samples can be determined based on the actual application. For example, in the advertising field, to train a conversion rate prediction model, positive samples refer to samples in which conversion behavior occurs after clicking on an ad, and negative samples refer to samples in which no conversion behavior occurs after clicking on an ad. Different ads may also correspond to different conversion behaviors. Specifically, conversion behaviors may include downloading, activation, consumption, etc.

[0046] In the advertising field, each training sample is an ad click log (the click time is represented by clickTime). The sample label (label) takes a value of 0 or 1, where 0 indicates that no conversion occurred after the click, and 1 indicates that a conversion occurred after the click. If the label is 1, the conversion return time (conversion time) is also provided.

[0047] S102: Inputting negative samples and time features into the scattering model to obtain the time distribution corresponding to the negative samples output by the scattering model.

[0048] The scattering model is pre-trained to scatter the temporal distribution of negative samples. The training data used in training the scattering model are the training negative samples and the training time features, where the training time features are determined based on the reflow time of the training positive samples.

[0049] Based on this, a scattering model is used to determine the time distribution of negative samples in the first training sample set, and the negative samples in the first training sample set and their time features are input into the scattering model. The time features are determined based on the reflow time of the positive samples in the first training sample set.

[0050] As can be seen from the above, the reflow time corresponding to positive samples in different conversion behaviors varies. For example, the reflow time corresponding to downloads and activations in shallow conversions is approximately one day, while the reflow time corresponding to consumption in deep conversions is approximately one week. To better adapt to the differences in reflow distribution across different conversion behaviors, a fragmented model is trained for conversion behaviors with the same or similar reflow distributions. For example, fragmented model 1 is trained for download or activation conversion behaviors with a reflow time of approximately one day, and fragmented model 2 is trained for consumption conversion behaviors. When the conversion behavior is download or activation, negative samples are input into fragmented model 1, and when the conversion behavior is consumption, negative samples are input into fragmented model 2.

[0051] The scattering model can be a wide&deep model, in which the wide part processes the temporal features and the deep part processes the negative samples. Figure 2a As shown, the deep portion of the negative sample input undergoes feature extraction and other steps before being fused with the temporal features at the output layer to determine the temporal distribution of the negative sample. The scattering model determines the distribution probability of negative samples in different time periods based on the characteristics and temporal features of the negative samples, and thus determines the temporal distribution of the negative samples based on the distribution probabilities. For example, the scattering model contains 246 time buckets, each corresponding to a different time period. The scattering model determines the probability of each negative sample in the 246 time buckets and determines the time bucket corresponding to the maximum probability value as the temporal distribution corresponding to the negative sample.

[0052] In this example, the time feature is taken into account when determining the time distribution of negative samples. Since the time feature is determined based on the reflow time of the positive examples, the scattering model will refer to the reflow time of the positive examples when determining the time distribution of negative samples. The negative samples will be scattered with reference to the reflow time of the positive examples to avoid the distribution of negative samples being too dispersed.

[0053] In one embodiment of the present disclosure, after obtaining the first training sample set, the method further includes: determining a time feature according to the reflow time of the positive sample.

[0054] In one embodiment of the present disclosure, the reflow time of the positive sample can be directly encoded to obtain the time feature. Alternatively, for any reflow time, multiple first parameters can be determined based on the reflow time and a sine function; the multiple first parameters are weighted and summed to obtain the time feature. Specifically, the following formula can be used:

[0055]

[0056] Among them, f(x) represents the time characteristic, F(x;i,j) represents the first parameter, x represents the reflow time, w ij Indicates the weight of the first parameter, ni and nj are positive integers, and the specific values ​​can be determined according to the actual situation. For example, when ni = 3 and nj = 4, it means that 12 first parameters can be determined, such as Figure 2b As shown, when x=10, there are 12 first parameters in total, and the time feature f is determined based on the 12 first parameters, where the value of the first parameter is the value indicated by each gray column; when x=18, there are 12 first parameters in total, and the time feature f is determined based on the 12 first parameters.

[0057] S103: Determine a second training sample set according to the time distribution.

[0058] After determining the time distribution corresponding to the negative samples, a second training sample set is determined based on the time distribution, where the ratio of positive samples to negative samples in the second training sample set meets a preset condition. That is, after determining the time distribution of the negative samples, positive samples that meet the aforementioned time distribution are determined based on the reflow time of each positive sample in the first training sample set, and the positive and negative samples that meet the aforementioned time distribution form the second training sample set.

[0059] It can be seen that after obtaining the first training sample set, the negative samples and time features in the first training sample set are input into the scattering model. Among them, the time feature is determined based on the reflow time of the positive samples in the first training sample set, and the scattering model is pre-trained to determine the temporal distribution of negative samples, and the time distribution of negative samples will be determined based on the negative samples and the time features. After determining the time distribution of the negative samples, the second training sample set will be determined based on the time distribution, so that the ratio of positive samples and negative samples in the second training sample set determined meets the preset conditions. Since the present application refers to the reflow time of the positive samples when determining the time distribution of the negative samples, the distribution of the negative samples can be accurately scattering, thereby improving the stability of the distribution ratio of positive and negative samples.

[0060] In one embodiment of the present disclosure, after obtaining the second training sample set, the CVR prediction model can be generated by training using the second training sample set. Specifically, Figure 3 As shown in the figure, a method for generating a conversion rate prediction model provided by an embodiment of the present application may include:

[0061] S301: Obtain a training sample set, in which the ratio of positive samples to negative samples meets a preset condition.

[0062] The training sample set is obtained according to the method described in S101-S104 above, and the distribution ratio of positive and negative samples in the training sample set meets the preset conditions. Positive samples are samples that result in conversion after click, and negative samples are samples that do not result in conversion after click.

[0063] S302: Train the initial model using the training sample set to generate a conversion rate estimation model.

[0064] In this embodiment, the initial model is trained using positive and negative samples from the training sample set. Once the training objectives are achieved, a conversion rate estimation model is obtained. Because the distribution ratio of positive and negative samples in the training sample set meets the preset conditions, the conversion rate estimation model generated by the training can achieve accurate estimates, avoiding underestimation in the early morning and overestimation in the evening.

[0065] Based on the above method embodiments, the embodiments of the present application provide corresponding devices and equipment, which will be described below with reference to the accompanying drawings.

[0066] See also Figure 4 , which is a structural diagram of a training sample acquisition device provided in an embodiment of the present application, such as Figure 4 As shown, the device 400 includes: a first acquiring unit 401 , a second acquiring unit 402 and a determining unit 403 .

[0067] A first acquisition unit 401 is configured to acquire a first training sample set, wherein the first training sample set includes positive samples and negative samples. The positive samples are samples in which conversion occurs after a click, and the negative samples are samples in which no conversion occurs after a click.

[0068] A second acquisition unit 402 is configured to input the negative sample and the time feature into a scattering model, and obtain a time distribution corresponding to the negative sample output by the scattering model, wherein the time feature is determined based on the reflow time of the positive sample, and the scattering model is pre-trained to determine the time distribution of the negative sample;

[0069] The determining unit 403 is configured to determine a second training sample set according to the time distribution, wherein the ratio of positive samples to negative samples in the second training sample set meets a preset condition.

[0070] In one embodiment of the present disclosure, the determining unit 403 is further configured to determine the time feature according to the reflow time of the positive sample.

[0071] In one embodiment of the present disclosure, the determination unit 403 is specifically used to encode the reflow time of the positive sample to obtain the time feature; or, for any reflow time, determine multiple first parameters based on the reflow time and the sine function; and perform weighted summation on the multiple first parameters to obtain the time feature.

[0072] In one embodiment of the present disclosure, the scattering model is related to the conversion behavior corresponding to the positive sample.

[0073] In one embodiment of the present disclosure, the converted behavior includes activation behavior, downloading behavior, or consumption behavior.

[0074] In one embodiment of the present disclosure, the fragmented model is a wide&deep model, in which the wide part processes temporal features and the deep part processes negative samples.

[0075] It should be noted that the implementation of each unit in this embodiment can refer to the relevant description in the above method embodiment, and this embodiment will not be repeated here.

[0076] See also Figure 5 , which is a structural diagram of a conversion rate estimation model generation device provided in an embodiment of the present application, such as Figure 5 As shown, the device 500 includes: an acquisition unit 501 and a generation unit 502.

[0077] The acquisition unit 501 is used to acquire a training sample set, wherein the training sample set is based on Figure 1 The ratio of positive samples to negative samples in the training sample set obtained by the method meets the preset conditions, the positive samples are samples in which conversion behavior occurs after being clicked, and the negative samples are samples in which no conversion behavior occurs after being clicked;

[0078] The generating unit 502 is configured to train the initial model using the training sample set to generate a conversion rate prediction model.

[0079] It should be noted that the specific implementation of each unit in this embodiment can refer to the relevant description in the above method embodiment.

[0080] The division of units in the embodiments of the present application is schematic and is only a logical functional division. In actual implementation, there may be other division methods. The functional units in the embodiments of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. For example, in the above embodiment, the processing unit and the sending unit can be the same unit or different units. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0081] See also Figure 6 , which shows a schematic structural diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0082] like Figure 6As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0083] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0084] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0085] The electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0086] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the method provided in the above embodiment is implemented.

[0087] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0088] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0089] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0090] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device can perform the method.

[0091] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit / module does not, in some cases, limit the unit itself.

[0094] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0095] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0096] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0097] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0098] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0099] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0100] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for obtaining training samples, characterized in that: The method comprises: Obtain a first training sample set, the first training sample set including positive samples and negative samples, the positive samples are samples in which conversion behavior occurs after being clicked, and the negative samples are samples in which no conversion behavior occurs after being clicked; Inputting the negative sample and the time feature into a scattering model, obtaining a time distribution corresponding to the negative sample output by the scattering model, wherein the time feature is determined based on the reflow time of the positive sample, and the scattering model is pre-trained to determine the time distribution of the negative sample; A second training sample set is determined according to the time distribution, and a ratio of positive samples to negative samples in the second training sample set meets a preset condition.

2. The method according to claim 1, characterized in that The time feature is obtained by: Encoding the reflow time of the positive sample to obtain the time feature; or, For any reflow time, determining a plurality of first parameters according to the reflow time and a sine function; Perform weighted summation on the multiple first parameters to obtain the time feature.

3. The method according to claim 1, characterized in that The scattering model is related to the conversion behavior corresponding to the positive sample.

4. The method according to claim 3, characterized in that The converted behaviors include activation behaviors, download behaviors or consumption behaviors.

5. The method according to claim 1, wherein The fragmented model is a wide&deep model, in which the wide part processes the temporal features and the deep part processes the negative samples.

6. A method for generating a conversion rate prediction model, characterized in that: The method comprises: Obtaining a training sample set, where the training sample set is obtained based on the method according to any one of claims 1 to 5, wherein the ratio of positive samples to negative samples in the training sample set meets a preset condition, the positive samples are samples in which conversion behavior occurs after being clicked, and the negative samples are samples in which no conversion behavior occurs after being clicked; The initial model is trained using the training sample set to generate a conversion rate estimation model.

7. A training sample acquisition device, characterized in that: The device comprises: A first acquisition unit is configured to acquire a first training sample set, wherein the first training sample set includes positive samples and negative samples, wherein the positive samples are samples in which conversion occurs after being clicked, and the negative samples are samples in which no conversion occurs after being clicked; a second acquisition unit, configured to input the negative sample and a time feature into a scattering model, and obtain a time distribution corresponding to the negative sample output by the scattering model, wherein the time feature is determined based on the reflow time of the positive sample, and the scattering model is pre-trained and configured to determine the time distribution of the negative sample; A determining unit is configured to determine a second training sample set according to the time distribution, wherein a ratio of positive samples to negative samples in the second training sample set meets a preset condition.

8. A conversion rate estimation model generation device, characterized in that: The device comprises: an acquisition unit, configured to acquire a training sample set, wherein the training sample set is acquired based on the method according to any one of claims 1 to 5, wherein the ratio of positive samples to negative samples in the training sample set meets a preset condition, wherein the positive samples are samples in which conversion behavior occurs after being clicked, and the negative samples are samples in which no conversion behavior occurs after being clicked; The generating unit is used to train the initial model using the training sample set to generate a conversion rate estimation model.

9. An electronic device, characterized in that: The device includes: a processor and a memory; The memory is used to store instructions or computer programs; The processor is configured to execute the instructions or computer program in the memory, so that the electronic device executes the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a device, the device is caused to execute the method according to any one of claims 1 to 6.

Images