A data processing method and related device

Abstract

Embodiments of the present application disclose a data processing method and related devices. The method includes: for each candidate advertisement corresponding to a target exposure request, obtaining the advertisement status corresponding to each candidate advertisement and the overall status of the advertisement placement platform that responds to the target exposure request; for each candidate advertisement, determining the probability that the candidate advertisement belongs to each reference advertisement type through a classification network in a scoring model; based on the probability that the candidate advertisement belongs to each reference advertisement type, through a scoring network in the scoring model, determining the competition score of the candidate advertisement for the target exposure request according to the advertisement status corresponding to the candidate advertisement and the overall status; the scoring model includes multiple scoring networks respectively corresponding to each reference advertisement type; determining a target advertisement exposed through the target exposure request according to the competition scores of each candidate advertisement for the target exposure request. This method can improve the accuracy of the scores configured by the scoring model for advertisements.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a data processing method and related devices. Background Art

[0002] In practical applications, when an advertiser places an advertisement on an advertising platform, targeting conditions will be set for the placed advertisement. For example, the exposure target of the advertisement is set to males under 30 years old in Shanghai, and so on. When the advertising platform detects an exposure request, it will recall the advertisements whose targeting conditions match the exposure request, and perform filtering processes such as rough ranking and fine ranking on the recalled advertisements to obtain a candidate advertisement queue corresponding to the exposure request; furthermore, score the advertisements in the candidate advertisement queue, and determine the advertisement to be exposed through this exposure request according to the scores of the advertisements in the candidate advertisement queue.

[0003] In related technologies, a model trained by a reinforcement learning algorithm is usually used to score the advertisements in the above candidate advertisement queue.

[0004] However, the inventors of this application have found through research that the above model usually has difficulty in accurately scoring various advertisements. The reason is that the advertisements placed on the advertising platform are rich and diverse. In order to adapt to this characteristic of the advertising platform, when training a model for scoring advertisements, the model is usually used to score a large number of different types of advertisements, which will make the model have a huge action space. This huge action space will cause the trained model to be difficult to converge, that is, the model performance cannot meet the expected requirements. Correspondingly, in practical applications, determining the finally exposed advertisement according to the scores configured for the advertisements by this model often makes it difficult for the advertising platform to generate ideal benefits. Summary of the Invention

[0005] Embodiments of this application provide a data processing method and related devices, which can improve the accuracy of the scores configured for advertisements by a scoring model, thereby helping to improve the overall revenue of the advertising platform.

[0006] In view of this, in a first aspect of this application, a data processing method is provided, and the method includes:

[0007] For each candidate advertisement corresponding to a target exposure request, obtain the advertisement state corresponding to each candidate advertisement, where the advertisement state is used to characterize the competition conditions when its corresponding candidate advertisement competes for the target exposure request; and obtain the overall state of the advertising platform that responds to the target exposure request, where the overall state is used to characterize the current completion status of the exposure task of the advertising platform;

[0008] For each of the candidate ads, determine the probability that the candidate ad belongs to each reference ad type through the classification network in the scoring model;

[0009] For each of the candidate ads, based on the probability that the candidate ad belongs to each reference ad type, through the scoring network in the scoring model, determine the competition score of the candidate ad for the target exposure request according to the ad status corresponding to the candidate ad and the overall status; the scoring model includes multiple scoring networks respectively corresponding to each of the reference ad types;

[0010] Determine the target ad to be exposed through the target exposure request according to the competition scores of each of the candidate ads for the target exposure request.

[0011] A second aspect of the present application provides a data processing device, the device includes:

[0012] A status acquisition module, configured to, for each candidate ad corresponding to a target exposure request, acquire the ad status corresponding to each candidate ad, where the ad status is used to characterize the competition conditions when its corresponding candidate ad competes for the target exposure request; and acquire the overall status of the ad placement platform in response to the target exposure request, where the overall status is used to characterize the current exposure task completion situation of the ad placement platform;

[0013] A classification module, configured to, for each of the candidate ads, determine the probability that the candidate ad belongs to each reference ad type through the classification network in the scoring model;

[0014] A scoring module, configured to, for each of the candidate ads, based on the probability that the candidate ad belongs to each reference ad type, through the scoring network in the scoring model, determine the competition score of the candidate ad for the target exposure request according to the ad status corresponding to the candidate ad and the overall status; the scoring model includes multiple scoring networks respectively corresponding to each of the reference ad types;

[0015] An ad selection module, configured to determine the target ad to be exposed through the target exposure request according to the competition scores of each of the candidate ads for the target exposure request.

[0016] A third aspect of the present application provides a computer device, the device includes a processor and a memory:

[0017] The memory is used to store a computer program;

[0018] The processor is configured to execute the steps of the data processing method as described in the first aspect above according to the computer program.

[0019] A fourth aspect of the present application provides a computer-readable storage medium for storing a computer program for executing the steps of the data processing method described in the first aspect above.

[0020] A fifth aspect of the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to execute the steps of the data processing method described in the first aspect above.

[0021] As can be seen from the above technical solutions, the embodiments of the present application have the following advantages:

[0022] The embodiments of the present application provide a data processing method. The method uses a scoring model including multiple scoring networks to score candidate advertisements corresponding to an exposure request. The multiple scoring networks in the scoring model are respectively applicable to scoring advertisements of different reference advertisement types. When using the scoring model to score candidate advertisements corresponding to a target exposure request, first, through a classification network in the scoring model, determine the probability that the candidate advertisement belongs to each reference advertisement type; then, based on the probability that the candidate advertisement belongs to each reference advertisement type, through a scoring network in the scoring model, according to the advertisement state corresponding to the candidate advertisement and the overall state of the advertisement placement platform, determine the competition score of the candidate advertisement for the target exposure request; furthermore, the target advertisement to be exposed through the target exposure request can be determined according to the competition scores of each candidate advertisement for the target exposure request. Since different scoring networks in the scoring model are applicable to scoring advertisements of different reference advertisement types, when training the scoring model, for each scoring network, only advertisements of the reference advertisement type applicable to it can be used to train it. In this way, the action space of each scoring network is not too large, and it is easier for the scoring network to converge in a smaller action space, that is, it is easier to make the trained scoring network have better performance. Correspondingly, the scoring model including each scoring network can also have higher performance and can accurately determine the corresponding scores for each candidate advertisement. Selecting the advertisement finally exposed by the advertisement placement platform based on the scores configured for the advertisements by the scoring model also helps the advertisement placement platform to obtain higher benefits. Description of the Drawings

[0023] Figure 1 It is a schematic diagram of an application scenario of the data processing method provided by the embodiment of the present application;

[0024] Figure 2 It is a schematic flowchart of the data processing method provided by the embodiment of the present application;

[0025] Figure 3 The working principle of the classification network provided by the embodiment of the present application;

[0026] Figure 4 A schematic diagram of the implementation of a scoring method of the scoring model provided by the embodiment of the present application;

[0027] Figure 5 A schematic diagram of the implementation of another scoring method of the scoring model provided by the embodiment of the present application;

[0028] Figure 6 A schematic diagram of the implementation of yet another scoring method of the scoring model provided by the embodiment of the present application;

[0029] Figure 7 A schematic diagram of the reinforcement learning structure provided by the embodiment of the present application;

[0030] Figure 8 A schematic diagram of the process of the scoring model training method provided by the embodiment of the present application;

[0031] Figure 9 A schematic diagram of the construction method and working method of the virtual advertising placement platform provided by the embodiment of the present application;

[0032] Figure 10 An exemplary bipartite graph provided by the embodiment of the present application;

[0033] Figure 11 A schematic diagram of the structure of a data processing device provided by the embodiment of the present application;

[0034] Figure 12 A schematic diagram of the structure of another data processing device provided by the embodiment of the present application;

[0035] Figure 13 A schematic diagram of the structure of the terminal device provided by the embodiment of the present application;

[0036] Figure 14 A schematic diagram of the structure of the server provided by the embodiment of the present application. Detailed implementation manners

[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0038] In the description and claims of this application and the above-mentioned drawings, terms such as "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0039] In the related art, when training a scoring model for scoring candidate advertisements using a reinforcement learning algorithm, in order to enable the scoring model to accurately score various advertisements, all advertisements that meet the targeting conditions of a certain exposure request are usually regarded as the training candidate advertisements corresponding to the exposure request. Furthermore, the scores corresponding to all training candidate advertisements are determined using the scoring model to be trained, and based on the scores corresponding to each training candidate advertisement, the advertisement to be exposed through the exposure request is selected from them. However, there are usually tens of thousands of advertisements that meet the targeting conditions of the exposure request. Configuring scores for tens of thousands of advertisements and selecting one final exposed advertisement from them will result in a huge action space for the scoring model to be trained, and a huge action space often makes it difficult for the scoring model to converge, resulting in poor performance of the finally trained scoring model and difficulty in accurately configuring scores for various advertisements.

[0040] To solve the technical problems existing in the above-mentioned related art, an embodiment of this application provides a data processing method.

[0041] In this data processing method, for each candidate advertisement corresponding to a target exposure request, first, the advertisement status corresponding to each candidate advertisement is obtained, and the advertisement status is used to characterize the competition conditions when its corresponding candidate advertisement competes for the target exposure request; and the overall status of the advertisement placement platform in response to the target exposure request is obtained, and the overall status is used to characterize the current completion status of the exposure task of the advertisement placement platform. Then, for each candidate advertisement, the probability that the candidate advertisement belongs to each reference advertisement type is determined through the classification network in the scoring model; furthermore, based on the probability that the candidate advertisement belongs to each reference advertisement type, through the scoring network in the scoring model, according to the advertisement status corresponding to the candidate advertisement and the overall status of the advertisement placement platform, the competition score of the candidate advertisement for the target exposure request is determined; the above-mentioned scoring model includes multiple scoring networks respectively corresponding to each reference advertisement type. Finally, based on the competition scores of each candidate advertisement for the target exposure request, the target advertisement to be exposed through the target exposure request is determined.

[0042] The above data processing method uses a scoring model including multiple scoring networks to score each candidate advertisement corresponding to a target exposure request, and the multiple scoring networks in the scoring model are respectively applicable to scoring advertisements of different reference advertisement types. Since different scoring networks in the scoring model are applicable to scoring advertisements of different reference advertisement types, when training the scoring model, each scoring network can be trained only with the advertisements of the reference advertisement type it is applicable to. In this way, the action space of each scoring network will not be too large, and it is easier for the scoring network to converge in a smaller action space, that is, it is easier to make the trained scoring network have better performance. Correspondingly, the scoring model including each scoring network can also have higher performance and can accurately determine the corresponding scores for each candidate advertisement. Selecting the advertisement finally exposed by the advertisement placement platform based on the scores configured for the advertisement by this scoring model also helps the advertisement placement platform to obtain higher revenue.

[0043] It should be understood that the data processing method provided in the embodiments of the present application can be applied to a computer device with data processing capabilities, and this computer device can be a terminal device or a server. Among them, the terminal device can specifically be a computer, a smart phone, a tablet computer, a personal digital assistant (Personal Digital Assitant, PDA), etc.; the server can specifically be an application server or a Web server. In actual deployment, it can be an independent server or a cluster server or a cloud server composed of multiple physical servers.

[0044] To facilitate understanding of the data processing method provided in the embodiments of the present application, the application scenario of this data processing method will be exemplarily introduced below by taking the execution subject of this data processing method as a server.

[0045] See Figure 1 , Figure 1 is a schematic diagram of the application scenario of the data processing method provided in the embodiments of the present application. As Figure 1 shown, this application scenario includes a terminal device 110, a server 120, and a database 130; the terminal device 110 and the server 120 can communicate through a network; the server 120 and the database 130 can also communicate through a network, or the database 130 can also be integrated in the server 120.

[0046] In the embodiments of the present application, the terminal device 110 faces the user and is used to display the exposed advertisement through a specific interface or window. The server 120 may be the background server of the advertisement placement platform, which is used to execute the data processing method provided by the embodiments of the present application, respond to the exposure request generated by the terminal device 110, and feedback the target advertisement exposed through the exposure request to the terminal device 110. The database 130 is used to store the advertisements placed by advertisers on the advertisement placement platform and the playback control parameters corresponding to the advertisements.

[0047] In practical applications, after the terminal device 110 detects that the user triggers an operation to open the advertisement playback interface or advertisement playback window, it can transmit a target exposure request to the server 120 through the network. For example, assume that the terminal device 110 detects that the user triggers an operation to open a certain video application, and the splash screen interface of the video application supports the exposure of advertisements. Then the terminal device 110 can send a target exposure request to the server 120, and the target exposure request can carry the corresponding targeting attributes of itself, such as the personal attributes of the user, etc.

[0048] After receiving the target exposure request sent by the terminal device 110, the server 120 can recall from the database 130 the advertisement whose corresponding targeting conditions match the targeting attributes of the target exposure request according to the targeting attributes corresponding to the target exposure request. For example, assume that the targeting attributes corresponding to the target exposure request indicate that the user is a male under 30 years old in Shanghai. Then the server 120 can recall from the database 130 the advertisement whose targeting conditions match "male under 30 years old in Shanghai". Furthermore, the server 120 can perform a series of screening and filtering processes such as rough ranking and fine ranking on the recalled advertisements, so as to obtain each candidate advertisement corresponding to the target exposure request.

[0049] For each candidate advertisement corresponding to the target exposure request, the server 120 can obtain the advertisement status corresponding to each candidate advertisement. Here, the advertisement status is used to characterize the competition conditions when its corresponding candidate advertisement competes for the target exposure request. Exemplarily, when the candidate advertisement is a contract advertisement, the server 120 can determine the competition environment of the contract advertisement according to the advertisement characteristics of other advertisements except the contract advertisement among the candidate advertisements; the server 120 can also obtain at least one piece of information such as the playback volume, shortage volume, scheduled playback volume, selling price, playback control parameters, and targeting conditions of the contract advertisement from the database 130; furthermore, splice the competition environment of the contract advertisement and the information related to the contract advertisement obtained from the database 130 to obtain the advertisement status corresponding to the contract advertisement. When the candidate advertisement is a competitive advertisement, the server 120 can determine the competition environment of the competitive advertisement according to the advertisement characteristics of other advertisements except the competitive advertisement among the candidate advertisements; furthermore, use the competition environment of the competitive advertisement as the advertisement status corresponding to the competitive advertisement.

[0050] In addition, the server 120 also needs to obtain the overall status of the advertising platform, which is used to characterize the completion status of the current exposure tasks of the advertising platform. Exemplarily, the server 120 can obtain the current overall advertising shortage, overcast volume, revenue, etc. of the advertising platform as the overall status of the advertising platform.

[0051] Furthermore, for each candidate advertisement corresponding to the target exposure request, the server 120 uses a pre-trained scoring model to determine its competition score for the target exposure request. Specifically, for each candidate advertisement, the probability that the candidate advertisement belongs to each reference advertisement type can be determined through the classification network 1211 in the scoring model 121; then, based on the probability that the candidate advertisement belongs to each reference advertisement type, through the scoring network 1212 in the scoring model 121, according to the advertisement status corresponding to the candidate advertisement and the overall status of the advertising platform, the competition score of the candidate advertisement for the target exposure request is determined.

[0052] It should be noted that the scoring model 121 includes multiple scoring networks 1212, and the multiple scoring networks 1212 are respectively applicable to score advertisements of different reference advertisement types. When training each scoring network 1212 in the scoring model 121, only the advertisements of the reference advertisement type applicable to the scoring network 1212 are used to train it. In this way, the action space of each scoring network 1212 will not be too large.

[0053] Finally, the server 120 can determine the target advertisement to be exposed through the target exposure request according to the competition scores of each candidate advertisement determined by the scoring model 121 for the target exposure request; and transmit the target advertisement to the terminal device 110 through the network, so that the terminal device 110 plays the target advertisement in the corresponding advertisement playing interface or advertisement playing window.

[0054] It should be understood that Figure 1 The application scenarios shown are only examples. In actual applications, the data processing method provided by the embodiments of the present application can also be applied to other scenarios. Here, no limitations are imposed on the application scenarios applicable to the data processing method provided by the embodiments of the present application.

[0055] The data processing method provided by the present application will be introduced in detail below through method embodiments.

[0056] See Figure 2 , Figure 2 which is a schematic flowchart of the data processing method provided by the embodiments of the present application. For the convenience of description, the following embodiments will still be introduced by taking the server as the execution subject of the data processing method. As Figure 2 shown, the data processing method includes the following steps:

[0057] Step 201: For each candidate advertisement corresponding to the target exposure request, obtain the advertisement status corresponding to each candidate advertisement, where the advertisement status is used to characterize the competition conditions when its corresponding candidate advertisement competes for the target exposure request; and obtain the overall status of the advertisement placement platform that responds to the target exposure request, where the overall status is used to characterize the current exposure task completion status of the advertisement placement platform.

[0058] In the embodiment of the present application, after the server detects the arrival of a target exposure request, it can determine each candidate advertisement corresponding to the target exposure request and obtain the advertisement status corresponding to each candidate advertisement; in addition, the server also needs to obtain the overall status of the advertisement placement platform that responds to the target exposure request.

[0059] In a possible implementation manner, the server can determine each candidate advertisement corresponding to the target exposure request in the following manner: directly determine each advertisement on the advertisement placement platform whose corresponding targeting condition matches the targeting attribute of the target exposure request as each candidate advertisement corresponding to the target exposure request. Or, recall each advertisement on the advertisement placement platform whose corresponding targeting condition matches the targeting attribute of the target exposure request, and perform rough ranking on the recalled advertisements, and use the advertisements retained after the rough ranking as each candidate advertisement corresponding to the target exposure request. Or, recall each advertisement on the advertisement placement platform whose corresponding targeting condition matches the targeting attribute of the target exposure request, and perform rough ranking and fine ranking on the recalled advertisements, and use the advertisements retained after the fine ranking as each candidate advertisement corresponding to the target exposure request.

[0060] It should be understood that in order to reduce the operation pressure on the server when scoring candidate advertisements, it is usually more inclined to select the advertisements retained after the fine ranking as the candidate advertisements corresponding to the target exposure request. Of course, in actual applications, the server can also use other methods to determine each candidate advertisement corresponding to the target exposure, and the present application does not make any limitations here.

[0061] In an embodiment of the present application, when the server determines the competitive score of the candidate advertisement for the target exposure request through the scoring model, at least two types of data need to be used, namely, the advertisement status corresponding to the candidate advertisement and the overall status of the advertisement delivery platform. Among them, the advertisement status corresponding to the candidate advertisement is used to characterize the competitive conditions when the candidate advertisement competes for the target exposure request; for example, the advertisement status can be used to characterize the competitive environment in which the corresponding candidate advertisement competes for the target exposure request. For example, the advertisement status can be determined according to the broadcast control parameters of the corresponding candidate advertisement, and the broadcast control parameters can reflect the competitiveness of the candidate advertisement to a certain extent. The overall status of the advertisement delivery platform is used to characterize the current exposure task completion status of the advertisement delivery platform. For example, the overall status of the advertisement delivery platform can include the current overall advertisement shortage of the advertisement delivery platform (that is, the difference between the current broadcast volume of the advertisement and its minimum broadcast volume in this cycle), advertisement overage (that is, the current broadcast volume of the advertisement exceeds the maximum broadcast volume of the advertisement in this cycle), revenue (that is, the revenue currently generated by broadcasting advertisements), and the like.

[0062] In a possible implementation, the candidate advertisements corresponding to the target exposure request may include at least one of contract advertisements and bidding advertisements. Among them, contract advertisements are advertisements generated in the following manner: the advertiser signs a contract with the advertising delivery platform, requiring the advertising delivery platform to play a predetermined amount of advertisements to the advertiser's specified type of users within a specified time. If the contract is reached, the advertiser needs to pay the corresponding advertising delivery fee to the advertising delivery platform. If the contract is not reached, that is, the actual amount of advertisements played does not reach its corresponding predetermined amount of advertisements, the advertising delivery platform needs to compensate the advertiser for a certain amount of fees. When playing such contract advertisements, if the actual amount of advertisements played exceeds its corresponding predetermined amount of advertisements, the advertising delivery platform will not charge additional fees. Bidding advertisements are a form of advertising that pays according to advertising effects (such as click-through rate, conversion rate, etc.); advertisers can give a bid for the advertisements they place. When an exposure request arrives, the bidding advertisements whose corresponding targeting conditions match the exposure request can compete for the exposure request based on the bids given in advance by the advertiser.

[0063] Generally, the candidate advertisements corresponding to the target exposure request may include both contract advertisements and bidding advertisements, that is, the embodiment of the present application is applied in the scenario of mixed contract advertisements and bidding advertisements; at this time, a corresponding method needs to be adopted to determine the corresponding advertisement status for the contract advertisements and bidding advertisements.

[0064] As an example, the advertisement state corresponding to the contract advertisement may include the competition environment when the contract advertisement competes for the target exposure request. This competition environment can be determined based on the advertisement features of other advertisements among the candidate advertisements except the contract advertisement itself. For example, the advertisement features of other advertisements among the candidate advertisements corresponding to the target exposure request except the contract advertisement itself can be concatenated to obtain the competition environment of the contract advertisement.

[0065] In addition, the advertisement state corresponding to the contract advertisement may further include at least one of the following information: the playback volume of the contract advertisement, the shortfall, the scheduled playback volume, the selling price, the playback control parameters, and the targeting conditions. Among them, the playback volume is the current playback volume of the contract advertisement. The shortfall is the difference in playback volume between the current playback volume of the contract advertisement and the minimum required playback volume of the contract advertisement within this period. The scheduled playback volume is the playback volume that the advertiser sets for the contract advertisement to reach when placing the contract advertisement. The selling price is the advertising placement price negotiated between the advertiser and the advertising placement platform when placing the contract advertisement. The playback control parameters may include, for example, Rate and Theta corresponding to the contract advertisement; Rate is a parameter used to control the playback of the contract advertisement, and Rate = 0.5 means that the contract advertisement has a 50% probability of entering the candidate advertisement queue; Theta is another parameter used to control the playback of the contract advertisement and is only used in the internal sorting of the contract advertisement. For example, if contract advertisement A and contract advertisement B match the same exposure request, the Theta of contract advertisement A is 0.3 and the Theta of contract advertisement B is 0.6, then the playback probability of contract advertisement A is 30% and the playback probability of contract advertisement B is 60%. Essentially, Theta is the ratio of the scheduled playback volume of the contract advertisement to the current inventory of the contract advertisement. The targeting conditions are the conditions that the exposure requests that can play the contract advertisement need to meet.

[0066] In the embodiments of the present application, the competition environment of the contract advertisement and the at least one piece of information related to the contract advertisement described above can be concatenated to obtain the advertisement state corresponding to the contract advertisement.

[0067] As an example, the advertisement state corresponding to the auction advertisement may include the competition environment when the auction advertisement competes for the target exposure request. This competition environment can be determined based on the advertisement features of other advertisements among the candidate advertisements except the auction advertisement itself. For example, the advertisement features of other advertisements among the candidate advertisements corresponding to the target exposure request except the auction advertisement itself can be concatenated to obtain the competition environment of the auction advertisement.

[0068] In the embodiments of the present application, the competition environment of the above-mentioned competitive advertisement can be directly used as the advertisement state corresponding to the competitive advertisement. Alternatively, at least one piece of information related to the competitive advertisement, such as the current revenue of the competitive advertisement, the targeting conditions, etc., can be obtained, and the competition environment of the above-mentioned competitive advertisement and at least one piece of information related to the competitive advertisement obtained are concatenated to obtain the advertisement state corresponding to the competitive advertisement.

[0069] It should be understood that in the embodiments of the present application, the candidate advertisements corresponding to the target exposure requests may further include other types of advertisements, and the advertisement states corresponding to the candidate advertisements may be determined according to other information related to the candidate advertisements, and the present application does not make any limitations here.

[0070] Step 202: For each of the candidate advertisements, determine the probability that the candidate advertisement belongs to each reference advertisement type through the classification network in the scoring model.

[0071] For each candidate advertisement, the server can use the classification network in the pre-trained scoring model to determine the probability that the candidate advertisement belongs to each reference advertisement type.

[0072] It should be noted that in the embodiments of the present application, advertisements can be divided into several reference advertisement types according to actual application requirements; for example, reference advertisement types can be divided according to whether the advertisement is out of stock, or reference advertisement types can be divided according to the user viewing frequency corresponding to the advertisement, etc., and the present application does not make any limitations on the reference advertisement types here.

[0073] In a possible implementation manner, the server can, through the classification network, determine the probability that the candidate advertisement belongs to each reference advertisement type according to the advertisement state corresponding to the candidate advertisement and the overall state of the advertisement placement platform.

[0074] Exemplarily, Figure 3 Figure (a) shows the working principle of the classification network in this implementation manner. As Figure 3As shown in (a), the server can splice the advertisement status corresponding to the candidate advertisement with the overall status of the advertisement placement platform; then, process the spliced status through the Multilayer Perceptron (MLP) layer in the classification network to obtain a Tensor; furthermore, classification processing can be performed based on this Tensor through the Softmax layer in the classification network, and a probability vector is output, which is used to represent the probability that the candidate advertisement belongs to each reference advertisement type. Suppose there are a total of four reference advertisement types, and the probability vector output by the classification network is [0.6, 0.1, 0.2, 0.1], indicating that the candidate advertisement has a 60% probability of belonging to the first reference advertisement type, a 10% probability of belonging to the second reference advertisement type, a 20% probability of belonging to the third reference advertisement type, and a 10% probability of belonging to the fourth reference advertisement type.

[0075] In another possible implementation, the server can determine the probability that the candidate advertisement belongs to each reference advertisement type through the classification network according to the advertisement status corresponding to the candidate advertisement.

[0076] Exemplarily, Figure 3 Figure (b) shows the working principle of the classification network in this implementation. As Figure 3 shown in Figure (b), the server can process the advertisement status corresponding to the candidate advertisement through the MLP layer in the classification network to obtain a Tensor; then, classification processing can be performed based on this Tensor through the Softmax layer in the classification network, and a probability vector is output, which is used to represent the probability that the candidate advertisement belongs to each reference advertisement type.

[0077] In yet another possible implementation, the server can determine the probability that the candidate advertisement belongs to each reference advertisement type through the classification network according to the advertisement features corresponding to the candidate advertisement.

[0078] Exemplarily, Figure 3 Figure (c) shows the working principle of the classification network in this implementation. As Figure 3 shown in Figure (c), the server can process the advertisement features corresponding to the candidate advertisement through the MLP layer in the classification network to obtain a Tensor. Here, the advertisement features can be determined according to the advertisement content of the candidate advertisement or according to the relevant playback parameters of the candidate advertisement (such as playback volume, scheduled playback volume, overplay volume, shortage volume, revenue, etc.); then, classification processing can be performed based on this Tensor through the Softmax layer in the classification network, and a probability vector is output, which is used to represent the probability that the candidate advertisement belongs to each reference advertisement type.

[0079] It should be understood that the working modes of the above three classification networks are only examples. In practical applications, other working modes can also be set for the classification network according to actual requirements, and the present application does not make any limitations here.

[0080] In practical applications, the above classification network can also be referred to as a gate network (Gate), which essentially corresponds to an attention mechanism (attention) layer and is used to control the features processed by the scoring network in the scoring model.

[0081] Step 203: For each of the candidate advertisements, based on the probabilities that the candidate advertisement belongs to each reference advertisement type, through the scoring network in the scoring model, according to the advertisement state corresponding to the candidate advertisement and the overall state, determine the competition score of the candidate advertisement for the target exposure request; the scoring model includes multiple scoring networks respectively corresponding to each of the reference advertisement types.

[0082] After determining the probabilities that the candidate advertisement belongs to each reference advertisement type through the classification network in the scoring model, based on the probabilities that the candidate advertisement belongs to each reference advertisement type, through the scoring network in the scoring model, according to the advertisement state corresponding to the candidate advertisement and the overall state of the advertisement placement platform, determine the competition score of the candidate advertisement for the target exposure request.

[0083] It should be noted that the scoring model provided in the embodiments of the present application includes multiple scoring networks (also referred to as expert networks), and there is a one-to-one correspondence between these multiple scoring networks and various reference advertisement types. For example, assuming there are four reference advertisement types in total, the scoring model includes four scoring networks. Each scoring network is applicable to score advertisements belonging to its corresponding reference advertisement type. For example, assuming the first scoring network is applicable to score advertisements of the first reference advertisement type, the score configured by the first scoring network for the advertisements belonging to the first reference advertisement type is more accurate than the scores configured by other scoring networks for this advertisement. The scoring model provided in the embodiments of the present application is trained based on a reinforcement learning mechanism, and the training method of the scoring model will be introduced in detail through another method embodiment below.

[0084] The inventors of the present application have found through research that if the number of scoring networks included in the scoring model is too large, it is likely that due to insufficient training samples for each scoring network, the scoring network is difficult to be fully trained, and at the same time, it will also increase the probability that the output dimension of the classification network in the scoring model is too large; if the number of scoring networks included in the scoring model is too small, it is close to the single-network structure in the related art, and the action space of each scoring network is still large. Based on this, it is necessary to set an appropriate number of scoring networks in the scoring model. Through research, it is found that setting four to eight scoring networks in the scoring model can achieve good results. Of course, the present application does not make any limitation on the number of scoring networks included in the scoring model.

[0085] In a possible implementation manner, when the server determines the competition score of the candidate advertisement for the target exposure request through the scoring network in the scoring model, it can be implemented in the following manner: Determine the input features of the candidate advertisement according to the advertisement state corresponding to the candidate advertisement and the overall state of the advertisement placement platform. Based on the probability that the candidate advertisement belongs to each reference advertisement type, perform weighted processing on the input features of the candidate advertisement to obtain the input features of the candidate advertisement under each reference advertisement type. Then, through each scoring network in the scoring model, according to the input features of the candidate advertisement under the reference advertisement type corresponding to the scoring network, configure a competition score for the candidate advertisement. Furthermore, according to the competition scores configured by each scoring network in the scoring model for the candidate advertisement, determine the competition score of the candidate advertisement for the target exposure request.

[0086] Exemplarily, Figure 4 shows the implementation process of this scoring method of the scoring model. As Figure 4As shown in the figure, the server can splice the advertisement status corresponding to the candidate advertisement with the overall status of the advertisement placement platform; then, process the spliced status through the MLP layer in the scoring model to obtain a Tensor as the input feature of the candidate advertisement. Then, the scoring model can perform weighted processing on the input feature based on the probabilities of the candidate advertisement belonging to each reference advertisement type to obtain the input features of the candidate advertisement under each reference advertisement type; for example, assuming that there are a total of four reference advertisement types, and the probabilities of the candidate advertisement belonging to these four reference advertisement types are 0.6, 0.1, 0.2, and 0.1 respectively, then the scoring model can multiply the input feature of the candidate advertisement by 0.6 to obtain the input feature of the candidate advertisement under the first reference advertisement type, multiply the input feature of the candidate advertisement by 0.1 to obtain the input feature of the candidate advertisement under the second reference advertisement type, multiply the input feature of the candidate advertisement by 0.2 to obtain the input feature of the candidate advertisement under the third reference advertisement type, and multiply the input feature of the candidate advertisement by 0.1 to obtain the input feature of the candidate advertisement under the fourth reference advertisement type. Furthermore, each scoring network in the scoring model can configure a competition score for the candidate advertisement according to the input feature of the candidate advertisement under the reference advertisement type corresponding to the scoring network; for example, the scoring network of the first reference advertisement type in the scoring model can configure a competition score for the candidate advertisement according to the input feature of the candidate advertisement under the first reference advertisement type, and the scoring network of the second reference advertisement type in the scoring model can configure a competition score for the candidate advertisement according to the input feature of the candidate advertisement under the second reference advertisement type, and so on. Finally, an average processing can be performed on the competition scores configured by each scoring network in the scoring model for the candidate advertisement to obtain the competition score of the candidate advertisement for the target exposure request.

[0087] In this way, all scoring networks in the scoring model determine the competition score of the candidate advertisement for the target exposure request based on the input features of different weights of the candidate advertisement, which can ensure the accuracy of the determined competition score.

[0088] In another possible implementation, when the server determines the competition score of the candidate advertisement for the target exposure request through the scoring network in the scoring model, it can be implemented in the following way: determine the input feature of the candidate advertisement according to the advertisement status corresponding to the candidate advertisement and the overall status of the advertisement placement platform. Then, through each scoring network in the scoring model, configure a competition score for the candidate advertisement according to the input feature of the candidate advertisement. Furthermore, based on the probabilities of the candidate advertisement belonging to each reference advertisement type, perform a weighted summation process on the competition scores configured by each scoring network in the scoring model for the candidate advertisement to obtain the competition score of the candidate advertisement for the target exposure request.

[0089] Exemplarily, Figure 5 the implementation process of this scoring method of the scoring model is shown. As Figure 5 shown, the server can splice the advertisement status corresponding to the candidate advertisement and the overall status of the advertisement placement platform; then, process the spliced status through the MLP layer in the scoring model to obtain a Tensor as the input feature of the candidate advertisement. Then, process the input feature of the candidate advertisement through each scoring network in the scoring model and output the competition score configured for the candidate advertisement. Furthermore, based on the probability that the candidate advertisement belongs to each reference advertisement type, the competition scores configured for the candidate advertisement by each scoring network are weighted and summed accordingly to obtain the competition score of the candidate advertisement for the target exposure request; for example, assuming that there are a total of four reference advertisement types, and the probabilities that the candidate advertisement belongs to these four reference advertisement types are 0.6, 0.1, 0.2, and 0.1 respectively, the scoring model can multiply the competition score configured by the scoring network corresponding to the first reference advertisement type by 0.6, multiply the competition score configured by the scoring network corresponding to the second reference advertisement type by 0.1, multiply the competition score configured by the scoring network corresponding to the third reference advertisement type by 0.2, multiply the competition score configured by the scoring network corresponding to the fourth reference advertisement type by 0.1, and then, add up the results of the above weighted processing to obtain the competition score of the candidate advertisement for the target exposure request.

[0090] In this way, all scoring networks in the scoring model configure competition scores for the candidate advertisement based on the input feature of the candidate advertisement, and then perform weighted summation processing on the competition scores configured by each scoring network, which can also ensure the accuracy of the determined competition score.

[0091] In another possible implementation manner, when the server determines the competition score of the candidate advertisement for the target exposure request through the scoring network in the scoring model, it can be implemented in the following manner: determine the input feature of the candidate advertisement according to the advertisement status corresponding to the candidate advertisement and the overall status of the advertisement placement platform. Then, based on the probability that the candidate advertisement belongs to each reference advertisement type, determine the scoring network in the scoring model that is most suitable for processing the candidate advertisement as the target scoring network. Furthermore, through the target scoring network, determine the competition score of the candidate advertisement for the target exposure request according to the input feature of the candidate advertisement.

[0092] Exemplarily, Figure 6 the implementation process of this scoring method of the scoring model is shown. As Figure 6As shown, the server can splice the ad status corresponding to the candidate ad with the overall status of the ad placement platform; then, process the spliced status through the MLP layer in the scoring model to obtain a Tensor as the input feature of the candidate ad. At the same time, the scoring model can also determine the target reference ad type to which the candidate ad belongs according to the probabilities of the candidate ad belonging to each reference ad type. For example, determine the maximum probability among the probabilities of the candidate ad belonging to each reference ad type, and then determine the reference ad type corresponding to the maximum probability as the target reference ad type to which the candidate ad belongs; correspondingly, the scoring model can determine the scoring network corresponding to the target reference ad type as the target scoring network. Figure 6 Taking the target scoring network as an example of the scoring network applicable to processing ads of the first reference ad type. Further, process the input feature of the candidate ad through the target scoring network in the scoring model to output the competition score of the candidate ad for the target exposure request.

[0093] In this way, selecting the most suitable scoring network for the candidate ad from the scoring model to score the candidate ad can, to a certain extent, ensure the accuracy of the determined competition score and reduce the computational resources required.

[0094] It should be understood that the implementation method of determining the competition score of the candidate ad for the target exposure request introduced above is only an example. In practical applications, the scoring model can also adopt other methods to use multiple scoring networks included therein to determine the competition score of the candidate ad for the target exposure request. This application does not make any limitations on this.

[0095] Step 204: Determine the target ad to be exposed through the target exposure request according to the competition scores of each of the candidate ads for the target exposure request.

[0096] After being processed by the scoring model, the server will obtain the competition scores of each candidate ad corresponding to the target exposure request for the target exposure request. Further, the server can determine the target ad finally exposed through the target exposure request according to the competition scores of each candidate ad for the target exposure request.

[0097] Exemplarily, the server can directly determine the candidate advertisement with the highest competition score for the target exposure request as the target advertisement to be exposed through the target exposure request. Alternatively, the server can obtain the advertisement competition scores corresponding to each candidate advertisement, where the advertisement competition score is determined based on the advertisement content of the candidate advertisement itself; then, for each candidate advertisement, determine the total competition score of the candidate advertisement according to the competition score of the candidate advertisement for the target exposure request and its corresponding advertisement competition score; finally, determine the candidate advertisement with the highest total competition score as the target advertisement to be exposed through the target exposure request. This application does not make any limitation on the method for determining the target advertisement to be exposed through the target exposure request.

[0098] The above data processing method uses a scoring model including multiple scoring networks to score each candidate advertisement corresponding to the target exposure request, and the multiple scoring networks in the scoring model are respectively applicable to score advertisements of different reference advertisement types. Since different scoring networks in the scoring model are applicable to score advertisements of different reference advertisement types, when training the scoring model, for each scoring network, it can be trained only using the advertisements of the reference advertisement type applicable to it. In this way, the action space of each scoring network will not be too large, and in a smaller action space, the scoring network is more likely to converge, that is, it is easier to make the trained scoring network have better performance. Correspondingly, the scoring model including each scoring network can also have higher performance and can accurately determine the corresponding scores for each candidate advertisement. Selecting the advertisement finally exposed by the advertisement placement platform based on the scores configured for the advertisement by this scoring model also helps the advertisement placement platform to obtain higher revenue.

[0099] The following uses method embodiments to Figure 2 introduce in detail the training method of the scoring model involved in the method embodiment shown. It should be noted that the scoring model in the embodiments of this application is trained based on the reinforcement learning mechanism. For the convenience of understanding, the reinforcement learning mechanism will be introduced below in combination with Figure 7 the schematic diagram of the AC (Actor-Critict) reinforcement learning structure shown.

[0100] The reinforcement learning mechanism explores the environment through the model, gives the scores of each optional strategy in the current environment state, and selects a strategy to execute based on the scores of various optional strategies. After executing this strategy, the environment state will change and a corresponding reward (positive reward or negative reward) will be generated, and this reward can provide a reference in the next round of strategy scoring process. The reinforcement learning aims to select the optimal strategy so that the environment state reaches the best after executing the optimal strategy.

[0101] In the application scenario of training a scoring model for scoring candidate advertisements corresponding to exposure requests, the Environment can train each training candidate advertisement corresponding to an exposure request. The scoring model to be trained (i.e., ActorNet) is responsible for scoring each training candidate advertisement corresponding to the training exposure request. Based on the scores of each training candidate advertisement, the training target advertisement (i.e., Action) to be exposed through the training exposure request is selected. After the training target advertisement is exposed, the State of the virtual advertising platform will change, and a reward corresponding to the advertisement exposure action can also be given. The Critic Net can give feedback information on the current scoring operation of the trained scoring model based on the State of the virtual advertising platform and the reward value. When the scoring model scores each training candidate advertisement corresponding to the training exposure request next time, this feedback information can be used as a reference.

[0102] See Figure 8 , Figure 8 is a schematic flowchart of the scoring model training method provided by an embodiment of the present application. For ease of description, the following embodiments still take the server as the execution subject of the scoring model training method as an example for introduction; it should be understood that the scoring model training method can also be executed by a terminal device in actual applications. As Figure 8 shown, the scoring model training method includes the following steps:

[0103] Step 801: Simulate a virtual advertising platform based on the historical data of the advertising platform.

[0104] In an embodiment of the present application, before the server trains the scoring model, it is necessary to first use the historical data of the advertising platform to simulate a virtual advertising platform to train the scoring model based on the environment of the virtual advertising platform.

[0105] In a possible implementation manner, the server can simulate the virtual advertising platform in the following way: Obtain the historical exposure request data, historical exposure log data, historical inventory data, and broadcast control parameters of the historical advertisements on the advertising platform. Based on the historical exposure request data and historical exposure log data, construct a training exposure request and determine each training candidate advertisement corresponding to the training exposure request. Based on the historical inventory data and the broadcast control parameters of the historical advertisements, determine the advertisement state corresponding to the training candidate advertisement. Based on the historical inventory data, historical exposure log data, and the broadcast control parameters of the historical advertisements, determine the overall state of the virtual advertising platform.

[0106] Figure 9 shows the construction method and working method of the virtual advertising platform provided by an embodiment of the present application. As Figure 9As shown in the figure, the construction of the virtual advertising platform is achieved through three stages: data source, data transmission, and data processing.

[0107] When the server constructs the virtual advertising platform, it can first obtain historical inventory data from the inventory system of the advertising platform, historical exposure log data and historical exposure request data from the log management system of the advertising platform, and playback control parameters of historical advertising from the playback control system of the advertising platform.

[0108] It should be noted that the inventory data stored in the inventory system usually comes from the inventory prediction service. The inventory prediction service is used to predict the future available inventory of advertisements using past advertising data, which can be accurate to the mapping between each exposure request and each advertisement, and can determine the inventory of each advertisement within a given time interval. The bipartite graph is calculated based on the inventory data. Through the bipartite graph, two very valuable data can be reflected: the playback probability of contract advertisements and the playback curve of the current period. The former can provide a reference for the advertising platform to ensure the quantity of contract advertisements, and the latter can provide the occupied space of contract advertisements for the advertising platform. Figure 10 As shown in the figure is an exemplary bipartite graph. Among them, the supply side is the inventory data, which can be expressed through the attribute dimension, and the demand side is the advertisement data, which can be expressed through the attribute dimension of the targeting condition. By associating the attribute dimension of the supply layer and the attribute dimension of the demand side, the mapping relationship between the inventory data and the advertisement data can be obtained.

[0109] In the embodiment of the present application, the advertisement state corresponding to the training candidate advertisement can be determined based on the historical inventory data obtained from the inventory system of the advertising platform. For example, when the training candidate advertisement is a contract advertisement, its corresponding short-play quantity, over-play quantity, etc. can be determined. The overall state of the simulated virtual advertising platform can also be determined based on the obtained historical inventory data, such as determining the overall short-play quantity, over-play quantity, etc. of the virtual advertising platform.

[0110] It should be noted that the exposure request data stored in the log management system are each historical exposure request generated on the terminal device side and its corresponding targeting attributes. The exposure log data stored in the log management system includes two types. One is the exposure log data at the request level, track_log, and the other is the exposure log data at the exposure level, joined_exposure. Among them, track_log includes the candidate ad queue corresponding to each exposure request after refined ranking, as well as the effective cost per mille (ecpm), predicted click-through rate (pctr), filtering conditions, support strategies, etc. of each competing ad in the candidate ad queue; joined_exposure includes the ad finally and actually exposed for each exposure request, as well as the billing information, ecpm information, etc. corresponding to the ad.

[0111] In the embodiment of the present application, a training exposure request can be constructed based on the historical exposure request data and historical exposure log data obtained from the log management system, and each training candidate ad corresponding to the training exposure request can be determined. It is also possible to determine the overall state of the virtual ad placement platform based on the obtained historical exposure log data.

[0112] It should be noted that the broadcast control parameters of the ads stored in the broadcast control system are parameters used to control the ad playback. For contract ads, its broadcast control parameters can be, for example, Rate, Theta, etc., which are used to assist in adjusting the playback situation of contract ads and are key information for ensuring the quantity of contract ads. For competing ads, its broadcast control parameters can be, for example, the bid price set by the advertiser for the ad, etc.

[0113] In the embodiment of the present application, the ad status corresponding to the training candidate ad corresponding to the training exposure request can be determined based on the broadcast control parameters obtained from the broadcast control system.

[0114] It should be understood that the above simulation method of the virtual ad placement platform is only an example. In actual applications, the server can also use other methods to simulate the virtual ad placement platform, and the present application does not limit this.

[0115] Step 802: For the training exposure request on the virtual ad placement platform, determine each training candidate ad corresponding to the training exposure request.

[0116] As introduced in step 801 above, when the server simulates the virtual ad placement platform, it can construct a training exposure request based on the obtained historical exposure request data; and determine each training candidate ad corresponding to the training exposure request based on the historical exposure log data.

[0117] In addition, the server also needs to determine the corresponding advertisement status for each training candidate advertisement. For example, based on the historical inventory data and its playback control parameters corresponding to the training candidate advertisement, the advertisement status corresponding to the training candidate advertisement is determined. The server also needs to determine the overall status of the virtual advertisement placement platform. For example, based on the acquired historical inventory data, historical exposure log data, and the playback control parameters of each historical placed advertisement, the current exposure task completion situation of the virtual advertisement placement platform is simulated, so as to determine the overall status of the virtual advertisement placement platform.

[0118] Step 803: Through the initial scoring model to be trained, according to the advertisement status corresponding to each of the training candidate advertisements and the overall status of the virtual advertisement placement platform, determine the training competition scores of each of the training candidate advertisements for the training exposure request; the initial scoring model includes an initial classification network and multiple initial scoring networks respectively corresponding to each reference advertisement type.

[0119] Furthermore, based on each training candidate advertisement corresponding to the training exposure request, the initial scoring model to be trained is trained. That is, for each training candidate advertisement, through the initial scoring model to be trained, according to the advertisement status corresponding to the training candidate advertisement and the overall status of the virtual advertisement placement platform, determine the training competition score of the training candidate advertisement for the training exposure request.

[0120] It should be understood that the initial scoring model trained in the embodiments of the present application is Figure 2 the same in both structure and working principle as the scoring model in the Figure 2 illustrated embodiment. For details, reference can be made to the relevant introduction content of the scoring network in the

[0121] illustrated embodiment. The initial scoring model includes an initial classification network and multiple initial scoring networks respectively corresponding to each reference advertisement type; among them, the initial classification network is used to determine the probability that the training candidate advertisement belongs to each reference advertisement type, and the initial scoring network is used to configure a training competition score for the training candidate advertisement according to the advertisement status corresponding to the training candidate advertisement and the overall status of the virtual advertisement placement platform.

[0122] Specifically, after the initial scoring model completes the scoring operation for each training candidate advertisement corresponding to the training exposure request in each round and selects the finally exposed advertisement based on the training competition scores of each training candidate advertisement for the training exposure request, the evaluation model will give feedback information on the scoring operation of the initial scoring model in this round according to the changes in the overall state of the virtual advertising platform and relevant reward values. This feedback information is used to reflect whether the scoring operation of the initial scoring model in this round is good or bad. It should be understood that if the feedback information reflects that the scoring operation of the initial scoring model in this round is good, it means that the advertisement exposure operation performed based on the scoring result of the scoring operation of the initial scoring model in this round makes the overall revenue of the virtual advertising platform tend to increase; if the feedback information reflects that the scoring operation of the initial scoring model in this round is bad, it means that the advertisement exposure operation performed based on the scoring result of the scoring operation of the initial scoring model in this round makes the overall revenue of the virtual advertising platform tend to decrease. When the initial scoring model re-scores each training candidate advertisement corresponding to the training exposure request in the next round, the feedback information can be input into the initial scoring model together with the advertisement state corresponding to the training candidate advertisement and the overall state of the virtual advertising platform.

[0123] In a possible implementation manner, when the server specifically trains each initial scoring network in the initial scoring model, for each training candidate advertisement, the probability that the training candidate advertisement belongs to each reference advertisement type can be determined through the initial classification network in the initial scoring model; then, according to the probability that the training candidate advertisement belongs to each reference advertisement type, the target reference advertisement type to which the training candidate advertisement belongs can be determined; furthermore, through the initial scoring network corresponding to the target reference advertisement type in the initial scoring model, according to the advertisement state corresponding to the training candidate advertisement, the overall state of the virtual advertising platform, and the reference feedback information, the training competition score of the training candidate advertisement for the training exposure request can be determined. Here, the reference feedback information is the feedback information given by the evaluation model for the scoring operation of the initial scoring network on each training candidate advertisement corresponding to the training exposure request last time.

[0124] Exemplarily, for a certain training candidate advertisement, the server can first splice together the advertisement status corresponding to the training candidate advertisement, the overall status of the virtual advertisement placement platform, and the reference feedback information, and process the spliced data through an MLP layer to obtain the input features of the training candidate advertisement. Then, the server can input the input features of the training candidate advertisement into the initial scoring model. After the initial scoring network in the initial scoring model processes the input features accordingly, it will output the probabilities that the training candidate advertisement belongs to each reference advertisement type. Then, the initial scoring model can determine the reference advertisement type to which the training candidate advertisement belongs as the target reference advertisement type based on the probabilities that the training candidate advertisement belongs to each reference advertisement type. Furthermore, the initial scoring model will call the initial scoring network corresponding to this target reference advertisement type, process the input features of the training candidate advertisement through this initial scoring network, and finally output the training competition score of the training candidate advertisement for this training exposure request.

[0125] In this way, after setting in advance the correspondence between the initial scoring network in the initial scoring model and the reference advertisement types, and determining the reference advertisement type to which a certain training candidate advertisement belongs through the initial scoring network in the initial scoring model, the initial scoring network corresponding to this reference advertisement type can be directly used to score the training candidate advertisement, so that each initial scoring network can focus on learning the features of the advertisements belonging to its corresponding reference advertisement type, realizing the specialization of each initial scoring network.

[0126] In another possible implementation manner, when the server specifically trains each initial scoring network in the initial scoring model, for each training candidate advertisement, it can determine the input features of the training candidate advertisement according to the advertisement status corresponding to the training candidate advertisement, the overall status of the virtual advertisement placement platform, and the reference feedback information. The reference feedback information here is the feedback information given by the evaluation model for the scoring operation of each training candidate advertisement corresponding to the training exposure request by the initial scoring network in the previous round. Then, through the initial classification network in the initial scoring model, it determines the probabilities that the training candidate advertisement belongs to each reference advertisement type, and based on the probabilities that the training candidate advertisement belongs to each reference advertisement type, it performs weighted processing on the input features of the training candidate advertisement to obtain the input features of the training candidate advertisement under each reference advertisement type. Furthermore, through each initial scoring network in the initial scoring model, according to the input features of the training candidate advertisement under each reference advertisement type, it determines the training competition score of the training candidate advertisement for the training exposure request.

[0127] Exemplarily, for a certain training candidate advertisement, the server may first splice together the advertisement status corresponding to the training candidate advertisement, the overall status of the virtual advertisement placement platform, and the reference feedback information, and process the spliced data through an MLP layer to obtain the input features of the training candidate advertisement. Then, the server may input the input features of the training candidate advertisement into the initial scoring model. After the initial scoring network in the initial scoring model processes the input features accordingly, it will output the probabilities that the training candidate advertisement belongs to each reference advertisement type. Then, the initial scoring model may perform weighted processing on the input features of the training candidate advertisement based on the probabilities that the training candidate advertisement belongs to each reference advertisement type to obtain the input features of the training candidate advertisement under various reference advertisement types. Furthermore, each initial scoring network in the initial scoring model may process the input features of the training candidate advertisement under its corresponding reference advertisement type to configure a training competition score for the training candidate advertisement. Finally, an average processing is performed on the training competition scores configured by each initial scoring network for the training candidate advertisement to obtain the competition score of the training candidate advertisement for the training exposure request.

[0128] Comparing this model training method with the method of only training a single network structure in the related art, assuming that a training exposure request corresponds to 10,000 training candidate advertisements. When using a single scoring network in the related art to score each training candidate advertisement, the scoring network needs to estimate 10,000 training competition scores and backpropagate the gradients. When there are two training candidate advertisements with large differences, it is very likely that one gradient is a very large positive number and the next gradient is a very large negative number, which makes the scoring network very oscillatory and unable to converge. After classification by the initial classification network in the embodiments of the present application, the classification probability can make the input features of advertisements that do not belong to the reference advertisement type applicable to a certain scoring network very small. Correspondingly, the competition scores output by them have a small impact on the overall competition score. On the contrary, the classification probability can also make the input features of advertisements that belong to the reference advertisement type applicable to a certain scoring network very large. In this way, the former has a small gradient and the latter has a large gradient, which can enable each scoring network to learn better about the reference advertisement type applicable to itself.

[0129] It should be understood that the working mode of the above initial scoring model is only an example. In practical applications, the initial scoring model may also work based on other working modes, and the present application does not make any limitations thereto.

[0130] Step 804: Determine the training target advertisement to be exposed through the training exposure request according to the training competition scores of each training candidate advertisement for the training exposure request, and simulate the training rewards that will be generated when the virtual advertisement placement platform exposes the training advertisement.

[0131] After the server determines the competition scores of each training candidate advertisement corresponding to the training exposure request through the initial scoring model, it can determine the training target advertisement to be exposed through the training exposure request according to the competition scores of each training candidate advertisement for the training exposure request.

[0132] Furthermore, it is possible to simulate the scenario of exposing the training target advertisement on the virtual advertising platform and accordingly determine the overall state of the virtual advertising platform after exposing the training target advertisement. For example, simulate the overall shortage, overcast, revenue, etc. of the virtual advertising platform after exposing the training target advertisement. Also, it is possible to simulate the training rewards that will be generated after the virtual advertising platform exposes the training target advertisement. For example, assuming that the virtual advertising platform hopes that the exposure rate of the advertisement is as high as possible, then if the training target advertisement exposed this time is an advertisement without overcast, a positive training reward can be given. On the contrary, if the training target advertisement exposed this time is an overcast advertisement, a negative training reward can be given.

[0133] In a possible implementation manner, the server can determine the training target advertisement to be exposed through the training exposure request in the following way: obtain the advertisement competition scores corresponding to each training candidate advertisement, where the advertisement competition score is determined according to the advertisement features of the corresponding training candidate advertisement; then, determine the training target advertisement according to the training competition scores of each training candidate advertisement for the training exposure request and the advertisement competition scores corresponding to each training candidate advertisement.

[0134] Such as Figure 9As shown in the figure, after the virtual advertising placement platform determines the competition scores of each training candidate advertisement corresponding to the training exposure request for the training exposure request through the initial scoring model, it can select the advertisement to be exposed through the training exposure request from each training candidate advertisement through the online system of the virtual advertising placement platform. The online system of the virtual advertising placement platform may include a Feature Server and a Mixer; among them, the Feature Server can obtain the competition scores of each training candidate advertisement corresponding to the training exposure request for the training exposure request, as well as the advertisement competition scores of each training candidate advertisement. Here, the advertisement competition score is determined according to the advertisement characteristics of its corresponding training candidate advertisement; then, the Mixer can obtain the advertisement competition scores corresponding to each training candidate advertisement and the competition scores of each training candidate advertisement for the training exposure request from the Feature Server. Furthermore, for each training candidate advertisement, according to its corresponding advertisement competition score and its competition score for the training exposure request, determine the total competition score of the training candidate advertisement. Finally, select the training candidate advertisement with the highest total competition score for exposure as the training target advertisement to be exposed through the training exposure request. After the virtual advertising placement platform completes the exposure of the training target advertisement, it can record the data related to this exposure operation in the log.

[0135] Step 805: Through the judgment model, according to the overall state of the virtual advertising placement platform and the training reward after the training target advertisement is exposed, determine the feedback information corresponding to the scoring operation of the initial scoring model in this round; the feedback information is used as reference information and input into the initial scoring model when the initial scoring model scores each training candidate advertisement corresponding to the training exposure request in the next round, so as to assist in adjusting the model parameters of the initial scoring model.

[0136] As introduced in step 803 above, after the virtual advertising placement platform completes each exposure operation of the training target advertisement, the server can input the overall state of the virtual advertising placement platform and the training reward after the training target advertisement is exposed into the judgment model. The judgment model processes the input data accordingly and outputs its feedback information on the scoring operation of the initial scoring model in this round. This feedback information is used to reflect whether the training target advertisement exposed based on the scoring result of the initial scoring model in this round has a positive or negative impact on the overall revenue of the virtual advertising placement platform. And this feedback information will be used as reference information and input into the initial scoring model when the initial scoring model scores each training candidate advertisement corresponding to the training exposure request in the next round, so as to assist in adjusting the model parameters of the initial scoring model and making the model performance of the initial scoring model tend to be better.

[0137] Step 806: When it is confirmed that the training end condition is met, determine the initial scoring model as the scoring model.

[0138] The server can repeatedly execute the above steps 802 to 805 based on each training exposure request. After completing a round of corresponding exposure operations for each training exposure request, the server can record the overall revenue situation of the virtual advertising platform at this time. In this way, after completing multiple rounds of corresponding exposure operations for each training exposure request and recording the overall revenue situation of the virtual advertising platform after each round of exposure operations, when it is determined that the overall revenue of the virtual advertising platform is basically stable and no longer increases significantly, it can be determined that the current training end condition is met, and the initial scoring model at this time can be determined as the scoring model that can be put into practical application, that is, Figure 2 the scoring model in the illustrated embodiment.

[0139] The embodiment of the present application provides Figure 2 a model training method for the scoring model in the illustrated embodiment. When training a scoring model including multiple scoring networks by this method, for each scoring network, only the advertisements of the reference advertisement types applicable to it can be used to train it, so as to ensure that the action space of each scoring network is not too large. In a smaller action space, the scoring network is more likely to converge, that is, it is easier to make the trained scoring network have better performance. Correspondingly, the scoring model including each scoring network can also have higher performance and can accurately determine the corresponding scores for each candidate advertisement.

[0140] The inventor of the present application put the advertisement exposure method provided in the embodiment of the present application into use in an actual advertisement placement platform and found that both the overall revenue situation of the advertisement placement platform and the ecpm of the competing advertisements have been significantly improved. The ecpm of the competing advertisements has increased by 4.2%, and the consumption has increased by 7.1%.

[0141] For the data processing method described above, the present application also provides a corresponding data processing device to enable the above data processing method to be applied and implemented in practice.

[0142] See Figure 11 , Figure 11 which is Figure 2 a schematic structural diagram of a data processing device 1100 corresponding to the data processing method shown above. As Figure 11 shown, the data processing device 1100 includes:

[0143] A status acquisition module 1101, configured to obtain, for each candidate advertisement corresponding to a target exposure request, an advertisement status corresponding to each of the candidate advertisements, where the advertisement status is used to characterize the competition condition when its corresponding candidate advertisement competes for the target exposure request; and obtain an overall status of an advertisement placement platform that responds to the target exposure request, where the overall status is used to characterize the current exposure task completion situation of the advertisement placement platform;

[0144] A classification module 1102, configured to, for each of the candidate advertisements, determine the probability that the candidate advertisement belongs to each reference advertisement type through a classification network in a scoring model;

[0145] A scoring module 1103, configured to, for each of the candidate advertisements, based on the probability that the candidate advertisement belongs to each reference advertisement type, through a scoring network in the scoring model, determine a competition score of the candidate advertisement for the target exposure request according to the advertisement status corresponding to the candidate advertisement and the overall status; the scoring model includes a plurality of the scoring networks respectively corresponding to each of the reference advertisement types;

[0146] An advertisement selection module 1104, configured to determine a target advertisement to be exposed through the target exposure request according to the competition scores of each of the candidate advertisements for the target exposure request.

[0147] Optionally, based on the data processing device shown in Figure 11 the scoring module 1103 is specifically configured to:

[0148] Determine an input feature of the candidate advertisement according to the advertisement status corresponding to the candidate advertisement and the overall status;

[0149] Based on the probability that the candidate advertisement belongs to each reference advertisement type, perform weighted processing on the input feature of the candidate advertisement to obtain an input feature of the candidate advertisement under each reference advertisement type;

[0150] Through each of the scoring networks in the scoring model, configure a competition score for the candidate advertisement according to the input feature of the candidate advertisement under the reference advertisement type corresponding to the scoring network;

[0151] Determine a competition score of the candidate advertisement for the target exposure request according to the competition scores configured for the candidate advertisement by each of the scoring networks in the scoring model.

[0152] Optionally, based on the data processing device shown in Figure 11 the scoring module 1103 is specifically configured to:

[0153] Determine the input features of the candidate advertisement according to the advertisement status corresponding to the candidate advertisement and the overall status;

[0154] For each scoring network in the scoring model, configure a competition score for the candidate advertisement according to the input features of the candidate advertisement;

[0155] Based on the probabilities that the candidate advertisement belongs to each reference advertisement type, perform a weighted sum processing on the competition scores configured for the candidate advertisement by each scoring network in the scoring model to obtain the competition score of the candidate advertisement for the target exposure request.

[0156] Optionally, based on Figure 11 the data processing device shown, the scoring module 1103 is specifically configured to:

[0157] Determine the input features of the candidate advertisement according to the advertisement status corresponding to the candidate advertisement and the overall status;

[0158] Based on the probabilities that the candidate advertisement belongs to each reference advertisement type, determine the scoring network in the scoring model that is most suitable for processing the candidate advertisement as the target scoring network;

[0159] Through the target scoring network, determine the competition score of the candidate advertisement for the target exposure request according to the input features of the candidate advertisement.

[0160] Optionally, based on Figure 11 the data processing device shown, the classification module 1102 is specifically configured to determine the probabilities that the candidate advertisement belongs to each reference advertisement type in any of the following ways:

[0161] Through the classification network, determine the probabilities that the candidate advertisement belongs to each reference advertisement type according to the advertisement status corresponding to the candidate advertisement and the overall status;

[0162] Through the classification network, determine the probabilities that the candidate advertisement belongs to each reference advertisement type according to the advertisement status corresponding to the candidate advertisement;

[0163] Through the classification network, determine the probabilities that the candidate advertisement belongs to each reference advertisement type according to the advertisement features corresponding to the candidate advertisement.

[0164] Optionally, based on Figure 11 the data processing device shown, the candidate advertisement includes at least one of a contract advertisement and a competitive bid advertisement;

[0165] The advertisement state corresponding to the contract advertisement includes the competition environment when the contract advertisement competes for the target exposure request, which is determined according to the advertisement characteristics of other advertisements except the contract advertisement among the candidate advertisements; the advertisement state corresponding to the contract advertisement further includes at least one of the following information: the playback volume, the shortfall, the scheduled playback volume, the selling price, the broadcast control parameters, and the targeting conditions of the contract advertisement;

[0166] The advertisement state corresponding to the auction advertisement includes the competition environment when the auction advertisement competes for the target exposure request, which is determined according to the advertisement characteristics of other advertisements except the auction advertisement among the candidate advertisements.

[0167] Optionally, based on the Figure 11 data processing device shown, refer to Figure 12 , Figure 12 which is a schematic structural diagram of another data processing device 1200 provided in an embodiment of the present application. As Figure 12 shown, the device further includes a model training module 1201; the model training module 1201 includes:

[0168] A platform simulation sub-module 1202, configured to simulate a virtual advertisement placement platform based on the historical data of the advertisement placement platform;

[0169] A training data determination sub-module 1203, configured to determine each training candidate advertisement corresponding to the training exposure request for the training exposure request on the virtual advertisement placement platform;

[0170] A model training sub-module 1204, configured to determine, through an initial scoring model to be trained, the training competition scores of each training candidate advertisement for the training exposure request according to the advertisement state corresponding to each training candidate advertisement and the overall state of the virtual advertisement placement platform; the initial scoring model includes an initial classification network and a plurality of initial scoring networks respectively corresponding to each reference advertisement type;

[0171] A simulated exposure sub-module 1205, configured to determine a training target advertisement exposed through the training exposure request according to the training competition scores of each training candidate advertisement for the training exposure request, and simulate the training reward generated by the virtual advertisement placement platform exposing the training target advertisement;

[0172] The evaluation sub-module 1206 is used to determine, through an evaluation model, feedback information corresponding to the scoring operation of the current round of the initial scoring model according to the overall state of the virtual advertising platform and the training reward after the training target advertisement is exposed; the feedback information is input into the initial scoring model as reference information when the initial scoring model scores each training candidate advertisement corresponding to the training exposure request in the next round, so as to assist in adjusting the model parameters of the initial scoring model;

[0173] The model acquisition sub-module 1207 is used to determine the initial scoring model as the scoring model when it is confirmed that the training end condition is met.

[0174] Optionally, based on the Figure 12 shown data processing device, the model training sub-module 1204 is specifically used for:

[0175] For each training candidate advertisement, through the initial classification network in the initial scoring model, determine the probability that the training candidate advertisement belongs to each reference advertisement type;

[0176] According to the probability that the training candidate advertisement belongs to each reference advertisement type, determine the target reference advertisement type to which the training candidate advertisement belongs;

[0177] Through the initial scoring network corresponding to the target reference advertisement type in the initial scoring model, according to the advertisement state corresponding to the training candidate advertisement, the overall state of the virtual advertising platform, and the reference feedback information, determine the training competition score of the training candidate advertisement for the training exposure request; the reference feedback information is the feedback information given by the evaluation model for the scoring operation of the initial scoring network on each training candidate advertisement corresponding to the training exposure request in the previous round.

[0178] Optionally, based on the Figure 12 shown data processing device, the model training sub-module 1204 is specifically used for:

[0179] For each training candidate advertisement, according to the advertisement state corresponding to the training candidate advertisement, the overall state of the virtual advertising platform, and the reference feedback information, determine the input features of the training candidate advertisement; the reference feedback information is the feedback information given by the evaluation model for the scoring operation of the initial scoring network on each training candidate advertisement corresponding to the training exposure request in the previous round;

[0180] Through the initial classification network in the initial scoring model, determine the probability that the training candidate advertisement belongs to each reference advertisement type;

[0181] Based on the probabilities that the training candidate ads belong to each reference ad type, perform weighted processing on the input features of the training candidate ads to obtain the input features of the training candidate ads under each reference ad type;

[0182] Through each of the initial scoring networks in the initial scoring model, determine the training competition scores of the training candidate ads for the training exposure request according to the input features of the training candidate ads under each reference ad type.

[0183] Optionally, based on the Figure 12 data processing device shown, the platform simulation sub-module 1202 is specifically configured to:

[0184] Obtain the historical exposure request data, historical exposure log data, historical inventory data, and broadcast control parameters of the historical ads of the ad placement platform;

[0185] Based on the historical exposure request data and the historical exposure log data, construct the training exposure request and determine each training candidate ad corresponding to the training exposure request;

[0186] Based on the historical inventory data and the broadcast control parameters of the historical ads, determine the ad status corresponding to the training candidate ads;

[0187] Based on the historical inventory data, the historical exposure log data, and the broadcast control parameters of the historical ads, determine the overall status of the virtual ad placement platform.

[0188] Optionally, based on the Figure 12 data processing device shown, the simulation exposure sub-module 1205 is specifically configured to:

[0189] Obtain the ad competition scores corresponding to each of the training candidate ads; the ad competition scores are determined according to the ad features of the corresponding training candidate ads;

[0190] Determine the training target ad according to the training competition scores of each of the training candidate ads for the training exposure request and the ad competition scores corresponding to each of the training candidate ads.

[0191] The above data processing device uses a scoring model including multiple scoring networks to score each candidate advertisement corresponding to a target exposure request, and the multiple scoring networks in the scoring model are respectively applicable to scoring advertisements of different reference advertisement types. Since different scoring networks in the scoring model are applicable to scoring advertisements of different reference advertisement types, when training the scoring model, for each scoring network, only the advertisements of the reference advertisement type applicable to it can be used to train it. In this way, the action space of each scoring network will not be too large, and it is easier for the scoring network to converge in a smaller action space, that is, it is easier to make the trained scoring network have better performance. Correspondingly, the scoring model including each scoring network can also have higher performance and can accurately determine the corresponding score for each candidate advertisement. Selecting the advertisement finally exposed on the advertisement placement platform based on the score configured for the advertisement by this scoring model also helps the advertisement placement platform to obtain higher revenue.

[0192] An embodiment of the present application also provides a computer device for advertisement exposure. This computer device can specifically be a terminal device or a server. Hereinafter, the terminal device and the server provided in the embodiment of the present application will be introduced from the perspective of hardware implementation.

[0193] See Figure 13 , Figure 13 is a schematic structural diagram of the terminal device provided in the embodiment of the present application. As Figure 13 shown, for the sake of convenience of description, only the parts related to the embodiment of the present application are shown. For the specific technical details not disclosed, please refer to the method part of the embodiment of the present application. This terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant, a point of sales (POS), an in-vehicle computer, etc. Taking the terminal as a computer as an example:

[0194] Figure 13 What is shown is a block diagram of a part of the structure of a computer related to the terminal provided in the embodiment of the present application. Referring to Figure 13 , the computer includes: a radio frequency (RF) circuit 1310, a memory 1320, an input unit 1330 (including a touch panel 1331 and other input devices 1332), a display unit 1340 (including a display panel 1341), a sensor 1350, an audio circuit 1360 (which can be connected to a speaker 1361 and a microphone 1362), a wireless fidelity (WiFi) module 1370, a processor 1380, and a power supply 1390 and other components. Those skilled in the art can understand that Figure 13 the computer structure shown in does not constitute a limitation on the computer, and may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0195] The memory 1320 can be used to store software programs and modules. The processor 1380 executes various functional applications and data processing of the computer by running the software programs and modules stored in the memory 1320. The memory 1320 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the computer (such as audio data, phone book, etc.). In addition, the memory 1320 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0196] The processor 1380 is the control center of the computer, connects various parts of the entire computer using various interfaces and lines, and executes various functions of the computer and processes data by running or executing the software programs and / or modules stored in the memory 1320, and by calling the data stored in the memory 1320. Optionally, the processor 1380 may include one or more processing units; preferably, the processor 1380 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 1380.

[0197] In the embodiment of the present application, the processor 1380 included in the terminal further has the following functions:

[0198] For each candidate advertisement corresponding to the target exposure request, obtain the advertisement state corresponding to each candidate advertisement, where the advertisement state is used to characterize the competition conditions when its corresponding candidate advertisement competes for the target exposure request; and obtain the overall state of the advertising platform that responds to the target exposure request, where the overall state is used to characterize the current exposure task completion situation of the advertising platform;

[0199] For each candidate advertisement, determine the probability that the candidate advertisement belongs to each reference advertisement type through the classification network in the scoring model;

[0200] For each candidate advertisement, based on the probability that the candidate advertisement belongs to each reference advertisement type, through the scoring network in the scoring model, according to the advertisement state corresponding to the candidate advertisement and the overall state, determine the competition score of the candidate advertisement for the target exposure request; the scoring model includes a plurality of scoring networks respectively corresponding to each reference advertisement type;

[0201] Determine a target advertisement to be exposed through the target exposure request according to the respective competition scores of the candidate advertisements for the target exposure request.

[0202] Optionally, the processor 1380 is further configured to execute the steps of any implementation manner of the data processing method provided in the embodiments of the present application.

[0203] See Figure 14 , Figure 14 FIG. is a schematic structural diagram of a server 1400 provided in an embodiment of the present application. The server 1400 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPUs) 1422 (for example, one or more processors) and a memory 1432, and one or more storage media 1430 for storing application programs 1442 or data 1444 (for example, one or more mass storage devices). Among them, the memory 1432 and the storage media 1430 may be transient storage or persistent storage. The program stored in the storage media 1430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Further, the central processor 1422 may be configured to communicate with the storage media 1430 and execute a series of instruction operations in the storage media 1430 on the server 1400.

[0204] The server 1400 may further include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input / output interfaces 1458, and / or one or more operating systems, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM and so on.

[0205] The steps performed by the server in the above embodiments may be based on the Figure 14 server structure shown.

[0206] Among them, the CPU 1422 is used to execute the following steps:

[0207] For each candidate advertisement corresponding to the target exposure request, obtain the respective advertisement status of each candidate advertisement, where the advertisement status is used to characterize the competition conditions when its corresponding candidate advertisement competes for the target exposure request; and obtain the overall status of the advertisement placement platform that responds to the target exposure request, where the overall status is used to characterize the current exposure task completion situation of the advertisement placement platform;

[0208] For each of the candidate advertisements, determine the probability that the candidate advertisement belongs to each reference advertisement type through the classification network in the scoring model;

[0209] For each of the candidate advertisements, based on the probability that the candidate advertisement belongs to each reference advertisement type, through the scoring network in the scoring model, determine the competition score of the candidate advertisement for the target exposure request according to the advertisement state corresponding to the candidate advertisement and the overall state; the scoring model includes a plurality of the scoring networks respectively corresponding to each of the reference advertisement types;

[0210] Determine the target advertisement to be exposed through the target exposure request according to the competition scores of each of the candidate advertisements for the target exposure request.

[0211] Optionally, the CPU 1422 can also be used to execute the steps of any implementation manner of the data processing method provided in the embodiments of the present application.

[0212] The embodiments of the present application further provide a computer-readable storage medium for storing a computer program, and the computer program is used to execute any implementation manner of the data processing method described in the foregoing embodiments.

[0213] The embodiments of the present application further provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes any implementation manner of the data processing method described in the foregoing embodiments.

[0214] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0215] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0216] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0217] In addition, each functional unit in various embodiments of the present application may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0218] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store computer programs.

[0219] It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (one) of the following" or similar expressions refer to any combination of these items, including any combination of single item (one) or plural items (ones). For example, at least one (one) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0220] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, characterized in that, The method includes: For each candidate advertisement corresponding to the target exposure request, obtain the advertisement status corresponding to each candidate advertisement, where the advertisement status is used to characterize the competition conditions when its corresponding candidate advertisement competes for the target exposure request; and obtain the overall status of the advertising platform that responds to the target exposure request, where the overall status is used to characterize the current exposure task completion situation of the advertising platform; For each of the candidate advertisements, determine the probability that the candidate advertisement belongs to each reference advertisement type through the classification network in the scoring model; For each scoring network in the scoring model, combine the probability that the candidate advertisement belongs to each reference advertisement type, and the advertisement status and the overall status corresponding to the candidate advertisement, to determine the competition score of the candidate advertisement for the target exposure request; the scoring model includes multiple scoring networks respectively corresponding to each of the reference advertisement types; Determine the target advertisement exposed through the target exposure request according to the competition scores of each candidate advertisement for the target exposure request.

2. The method according to claim 1, characterized in that, The step of, for each scoring network in the scoring model, combining the probability that the candidate advertisement belongs to each reference advertisement type, and the advertisement status and the overall status corresponding to the candidate advertisement, to determine the competition score of the candidate advertisement for the target exposure request includes: Determine the input features of the candidate advertisement according to the advertisement status and the overall status corresponding to the candidate advertisement; Based on the probability that the candidate advertisement belongs to each reference advertisement type, perform a weighted processing on the input features of the candidate advertisement to obtain the input features of the candidate advertisement under each reference advertisement type; Through each scoring network in the scoring model, configure a competition score for the candidate advertisement according to the input features of the candidate advertisement under the reference advertisement type corresponding to the scoring network; Determine the competition score of the candidate advertisement for the target exposure request according to the competition scores configured for the candidate advertisement by each scoring network in the scoring model.

3. The method according to claim 1, wherein The step of, for each scoring network in the scoring model, combining the probability that the candidate advertisement belongs to each reference advertisement type, and the advertisement status and the overall status corresponding to the candidate advertisement, to determine the competition score of the candidate advertisement for the target exposure request includes: Determine the input features of the candidate advertisement according to the advertisement status and the overall status corresponding to the candidate advertisement; Through each scoring network in the scoring model, configure a competition score for the candidate advertisement according to the input features of the candidate advertisement; Based on the probability that the candidate advertisement belongs to each reference advertisement type, perform a weighted summation processing on the competition scores configured for the candidate advertisement by each scoring network in the scoring model to obtain the competition score of the candidate advertisement for the target exposure request.

4. The method according to claim 1, characterized in that For each scoring network in the scoring model, combining the probability that the candidate advertisement belongs to each reference advertisement type, and the advertisement state and the overall state corresponding to the candidate advertisement, to determine the competition score of the candidate advertisement for the target exposure request, includes: Determine the input features of the candidate advertisement according to the advertisement state and the overall state corresponding to the candidate advertisement; Based on the probability that the candidate advertisement belongs to each reference advertisement type, determine the scoring network in the scoring model that is most suitable for processing the candidate advertisement as the target scoring network; Through the target scoring network, determine the competition score of the candidate advertisement for the target exposure request according to the input features of the candidate advertisement.

5. The method according to claim 1, characterized in that Determining the probability that the candidate advertisement belongs to each reference advertisement type through the classification network in the scoring model includes any one of the following: Through the classification network, determine the probability that the candidate advertisement belongs to each reference advertisement type according to the advertisement state and the overall state corresponding to the candidate advertisement; Through the classification network, determine the probability that the candidate advertisement belongs to each reference advertisement type according to the advertisement state corresponding to the candidate advertisement; Through the classification network, determine the probability that the candidate advertisement belongs to each reference advertisement type according to the advertisement features corresponding to the candidate advertisement.

6. The method according to any one of claims 1 to 5, characterized in that, The candidate advertisement includes at least one of a contract advertisement and a competitive advertisement; The advertisement state corresponding to the contract advertisement includes the competition environment when the contract advertisement competes for the target exposure request, which is determined according to the advertisement features of other advertisements except the contract advertisement among the candidate advertisements; the advertisement state corresponding to the contract advertisement further includes at least one of the following information: the playback volume, the shortage volume, the scheduled playback volume, the selling price, the broadcast control parameters, and the targeting conditions of the contract advertisement; The advertisement state corresponding to the competitive advertisement includes the competition environment when the competitive advertisement competes for the target exposure request, which is determined according to the advertisement features of other advertisements except the competitive advertisement among the candidate advertisements.

7. The method according to claim 1, characterized in that, The scoring model is trained in the following manner: Based on the historical data of the advertisement placement platform, simulate a virtual advertisement placement platform; For the training exposure requests on the virtual advertisement placement platform, determine the respective training candidate advertisements corresponding to the training exposure requests; Through the initial scoring model to be trained, according to the advertisement state corresponding to each training candidate advertisement and the overall state of the virtual advertisement placement platform, determine the training competition scores of each training candidate advertisement for the training exposure request; the initial scoring model includes an initial classification network and multiple initial scoring networks respectively corresponding to each reference advertisement type; According to the training competition scores of each training candidate advertisement for the training exposure request, determine the training target advertisement exposed through the training exposure request, and simulate the training reward generated by exposing the training target advertisement on the virtual advertisement placement platform. Through the evaluation model, determine the feedback information corresponding to the scoring operation of this round of the initial scoring model according to the overall state of the virtual advertising platform and the training rewards after the training target advertisement is exposed; When the initial scoring model scores each training candidate advertisement corresponding to the training exposure request in the next round, the feedback information is input into the initial scoring model as reference information to assist in adjusting the model parameters of the initial scoring model; When it is confirmed that the training end condition is met, determine the initial scoring model as the scoring model.

8. The method according to claim 7, characterized in that Through the initial scoring model to be trained, according to the advertisement state corresponding to each training candidate advertisement and the overall state of the virtual advertising platform, determine the training competition score of each training candidate advertisement for the training exposure request, including: For each training candidate advertisement, through the initial classification network in the initial scoring model, determine the probability that the training candidate advertisement belongs to each reference advertisement type; According to the probability that the training candidate advertisement belongs to each reference advertisement type, determine the target reference advertisement type to which the training candidate advertisement belongs; Through the initial scoring network corresponding to the target reference advertisement type in the initial scoring model, according to the advertisement state corresponding to the training candidate advertisement, the overall state of the virtual advertising platform, and the reference feedback information, determine the training competition score of the training candidate advertisement for the training exposure request; the reference feedback information is the feedback information given by the evaluation model for the scoring operation of the initial scoring network on each training candidate advertisement corresponding to the training exposure request in the previous round.

9. The method according to claim 7, wherein Through the initial scoring model to be trained, according to the advertisement state corresponding to each training candidate advertisement and the overall state of the virtual advertising platform, determine the training competition score of each training candidate advertisement for the training exposure request, including: For each training candidate advertisement, according to the advertisement state corresponding to the training candidate advertisement, the overall state of the virtual advertising platform, and the reference feedback information, determine the input features of the training candidate advertisement; the reference feedback information is the feedback information given by the evaluation model for the scoring operation of the initial scoring network on each training candidate advertisement corresponding to the training exposure request in the previous round; Through the initial classification network in the initial scoring model, determine the probability that the training candidate advertisement belongs to each reference advertisement type; Based on the probability that the training candidate advertisement belongs to each reference advertisement type, perform weighted processing on the input features of the training candidate advertisement to obtain the input features of the training candidate advertisement under each reference advertisement type; Through each initial scoring network in the initial scoring model, according to the input features of the training candidate advertisement under each reference advertisement type, determine the training competition score of the training candidate advertisement for the training exposure request.

10. The method according to claim 7, wherein Simulating a virtual advertising platform based on the historical data of the advertising platform includes: Obtain the historical exposure request data, historical exposure log data, historical inventory data, and broadcast control parameters of the historical placed advertisements of the advertisement placement platform; Based on the historical exposure request data and the historical exposure log data, construct the training exposure request and determine each training candidate advertisement corresponding to the training exposure request; Based on the historical inventory data and the broadcast control parameters of the historical placed advertisements, determine the advertisement status corresponding to the training candidate advertisement; Based on the historical inventory data, the historical exposure log data, and the broadcast control parameters of the historical placed advertisements, determine the overall status of the virtual advertisement placement platform.

11. The method according to claim 7, wherein The determining the training target advertisement exposed by the training exposure request according to the training competition scores of each training candidate advertisement for the training exposure request includes: Obtain the advertisement competition score corresponding to each training candidate advertisement; the advertisement competition score is determined according to the advertisement characteristics of the corresponding training candidate advertisement; Determine the training target advertisement according to the training competition scores of each training candidate advertisement for the training exposure request and the advertisement competition scores corresponding to each training candidate advertisement.

12. A data processing device, characterized in that, The device includes: A status acquisition module, configured to, for each candidate advertisement corresponding to a target exposure request, obtain the advertisement status corresponding to each candidate advertisement, where the advertisement status is used to characterize the competition conditions when its corresponding candidate advertisement competes for the target exposure request; and obtain the overall status of the advertisement placement platform in response to the target exposure request, where the overall status is used to characterize the completion situation of the current exposure task of the advertisement placement platform; A classification module, configured to, for each candidate advertisement, determine the probability that the candidate advertisement belongs to each reference advertisement type through a classification network in a scoring model; A scoring module, configured to, for each scoring network in the scoring model, combine the probability that the candidate advertisement belongs to each reference advertisement type, the advertisement status corresponding to the candidate advertisement, and the overall status, and determine the competition score of the candidate advertisement for the target exposure request; the scoring model includes multiple scoring networks respectively corresponding to each reference advertisement type; An advertisement selection module, configured to determine the target advertisement exposed by the target exposure request according to the competition scores of each candidate advertisement for the target exposure request.

13. The device according to claim 12, characterized in that, The scoring module is specifically configured to: Determine the input features of the candidate advertisement according to the advertisement status corresponding to the candidate advertisement and the overall status; Based on the probability that the candidate advertisement belongs to each reference advertisement type, perform weighted processing on the input features of the candidate advertisement to obtain the input features of the candidate advertisement under each reference advertisement type; Through each scoring network in the scoring model, configure a competition score for the candidate advertisement according to the input features of the candidate advertisement under the reference advertisement type corresponding to the scoring network; Determine the competition score of the candidate advertisement for the target exposure request according to the competition scores configured for the candidate advertisement by each scoring network in the scoring model.

14. The device according to claim 12, characterized in that, The scoring module is specifically configured to: Determine the input features of the candidate advertisement according to the advertisement status corresponding to the candidate advertisement and the overall status; Configure a competition score for the candidate advertisement according to the input features of the candidate advertisement through each scoring network in the scoring model; Based on the probabilities that the candidate advertisement belongs to each reference advertisement type, perform a weighted summation process on the competition scores configured for the candidate advertisement by each scoring network in the scoring model to obtain the competition score of the candidate advertisement for the target exposure request.

15. The device according to claim 12, characterized in that, The scoring module is specifically configured to: Determine the input features of the candidate advertisement according to the advertisement status corresponding to the candidate advertisement and the overall status; Based on the probabilities that the candidate advertisement belongs to each reference advertisement type, determine the scoring network in the scoring model that is most suitable for processing the candidate advertisement as the target scoring network; Determine the competition score of the candidate advertisement for the target exposure request according to the input features of the candidate advertisement through the target scoring network.

16. The device according to claim 12, characterized in that, The classification module is specifically configured to determine the probabilities that the candidate advertisement belongs to each reference advertisement type in any of the following ways: Determine the probabilities that the candidate advertisement belongs to each reference advertisement type according to the advertisement status corresponding to the candidate advertisement and the overall status through the classification network; Determine the probabilities that the candidate advertisement belongs to each reference advertisement type according to the advertisement status corresponding to the candidate advertisement through the classification network; Determine the probabilities that the candidate advertisement belongs to each reference advertisement type according to the advertisement features corresponding to the candidate advertisement through the classification network.

17. The device according to any one of claims 12 to 16, characterized in that, The candidate advertisement includes at least one of a contract advertisement and a competitive advertisement; The advertisement status corresponding to the contract advertisement includes the competition environment when the contract advertisement competes for the target exposure request, which is determined according to the advertisement features of other advertisements except the contract advertisement among the candidate advertisements; the advertisement status corresponding to the contract advertisement further includes at least one of the following information: the playback volume, the shortage volume, the scheduled playback volume, the selling price, the broadcast control parameters, and the targeting conditions of the contract advertisement; The advertisement status corresponding to the competitive advertisement includes the competition environment when the competitive advertisement competes for the target exposure request, which is determined according to the advertisement features of other advertisements except the competitive advertisement among the candidate advertisements.

18. The device according to claim 12, characterized in that, The device further includes a model training module; the model training module includes: A platform simulation sub-module for simulating a virtual advertisement placement platform based on the historical data of the advertisement placement platform; A training data determination sub-module for determining each training candidate advertisement corresponding to the training exposure request for the training exposure request on the virtual advertisement placement platform; A model training sub-module for determining the training competition scores of each training candidate advertisement for the training exposure request according to the advertisement status corresponding to each training candidate advertisement and the overall status of the virtual advertisement placement platform through an initial scoring model to be trained; the initial scoring model includes an initial classification network and a plurality of initial scoring networks respectively corresponding to each reference advertisement type; The simulation exposure sub-module is used to determine a training target advertisement to be exposed by the training exposure request according to the training competition scores of each of the training candidate advertisements for the training exposure request, and simulate the training rewards that would be generated by exposing the training target advertisement on the virtual advertisement delivery platform; The evaluation sub-module is used to, through an evaluation model, determine feedback information corresponding to the scoring operation of this round of the initial scoring model according to the overall state of the virtual advertisement delivery platform and the training rewards after exposing the training target advertisement; the feedback information is input into the initial scoring model as reference information when the initial scoring model scores each training candidate advertisement corresponding to the training exposure request in the next round, so as to assist in adjusting the model parameters of the initial scoring model; The model acquisition sub-module is used to determine the initial scoring model as the scoring model when it is confirmed that the training end condition is met.

19. The device according to claim 18, characterized in that, The model training sub-module is specifically used for: For each of the training candidate advertisements, through the initial classification network in the initial scoring model, determine the probabilities of the training candidate advertisements belonging to each reference advertisement type; According to the probabilities of the training candidate advertisements belonging to each reference advertisement type, determine the target reference advertisement type to which the training candidate advertisements belong; Through the initial scoring network corresponding to the target reference advertisement type in the initial scoring model, according to the advertisement state corresponding to the training candidate advertisement, the overall state of the virtual advertisement delivery platform, and the reference feedback information, determine the training competition score of the training candidate advertisement for the training exposure request; The reference feedback information is the feedback information given by the evaluation model for the scoring operation of the initial scoring network on each training candidate advertisement corresponding to the training exposure request in the previous round.

20. The device according to claim 18, wherein The model training sub-module is specifically used for: For each of the training candidate advertisements, determine the input features of the training candidate advertisement according to the advertisement state corresponding to the training candidate advertisement, the overall state of the virtual advertisement delivery platform, and the reference feedback information; The reference feedback information is the feedback information given by the evaluation model for the scoring operation of the initial scoring network on each training candidate advertisement corresponding to the training exposure request in the previous round; Through the initial classification network in the initial scoring model, determine the probabilities of the training candidate advertisements belonging to each reference advertisement type; Based on the probabilities of the training candidate advertisements belonging to each reference advertisement type, perform weighted processing on the input features of the training candidate advertisement to obtain the input features of the training candidate advertisement under each reference advertisement type; Through each of the initial scoring networks in the initial scoring model, determine the training competition score of the training candidate advertisement for the training exposure request according to the input features of the training candidate advertisement under each reference advertisement type.

21. The device according to claim 18, characterized in that, The platform simulation sub-module is specifically used for: Obtain the historical exposure request data, historical exposure log data, historical inventory data, and broadcast control parameters of historical advertisements of the advertisement delivery platform; Construct the training exposure request based on the historical exposure request data and the historical exposure log data, and determine each training candidate advertisement corresponding to the training exposure request; Determine the advertisement status corresponding to the training candidate advertisement based on the historical inventory data and the broadcast control parameters of the historical delivered advertisement; Determine the overall status of the virtual advertisement delivery platform based on the historical inventory data, the historical exposure log data, and the broadcast control parameters of the historical delivered advertisement.

22. The device according to claim 18, characterized in that, The simulation exposure sub-module is specifically configured to: Obtain the advertisement competition score corresponding to each of the training candidate advertisements; the advertisement competition score is determined according to the advertisement features of the corresponding training candidate advertisement; Determine the training target advertisement according to the training competition score of each of the training candidate advertisements for the training exposure request and the advertisement competition score corresponding to each of the training candidate advertisements.

23. A computer device, characterized in that, The device includes a processor and a memory; The memory is used to store a computer program; The processor is configured to execute the data processing method according to any one of claims 1 to 11 based on the computer program.

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the data processing method according to any one of claims 1 to 11.

25. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or the instruction is executed by the processor, the data processing method according to any one of claims 1 to 11 is implemented.

Images