Advertisement experiment method, device, equipment, storage medium, product and system

Abstract

The invention provides an advertisement experiment method, device and equipment, a storage medium, a product and a system, and the method comprises the steps: creating a plurality of experiment advertisements according to experiment elements of an advertisement putting experiment, and the experiment elements comprise at least one experiment variable and a plurality of parameter groups of each experiment variable; according to a preset flow distribution strategy, a first advertisement putting strategy corresponding to the first flow request is determined in the advertisement putting strategies corresponding to the experiment advertisements, and the first flow request is used for triggering advertisement putting; and putting a first experimental advertisement corresponding to the first advertisement putting strategy, and collecting experimental advertisement putting data corresponding to the first traffic request, the experimental advertisement putting data representing an advertisement effect. According to the method and the device, the efficiency and the accuracy of the advertisement experiment can be effectively improved, manual errors are reduced, the fairness of the experiment environment is ensured, and more credible advertisement experiment data can be provided for optimizing an advertisement putting strategy.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an advertising experiment method, device, equipment, storage medium, product and system. Background Art

[0002] In the current digital age, advertising has become an important means for companies to promote their products, services and brands. With the rapid development of Internet technology, the forms and channels of advertising have become increasingly diversified. How to effectively evaluate and optimize advertising strategies to improve advertising effectiveness and reduce costs has become a common focus for advertisers and advertising platforms.

[0003] The existing advertising experiment method usually requires operators to create multiple experimental ads, each of which contains different sets of advertising strategy parameters, so that the effect of each parameter group can be tested separately while controlling other variables. This process is extremely cumbersome, and it cannot guarantee that all ads are in a fair comparison environment throughout the process, nor can it avoid various errors caused by manual operations. Summary of the invention

[0004] In view of this, the embodiments of the present application are committed to providing an advertising experiment method, device, equipment, storage medium, product and system, which can effectively improve the efficiency and accuracy of advertising experiments, reduce human errors, and ensure the fairness of the experimental environment by automatically creating multiple experimental advertisements and delivering experimental advertisements based on preset traffic allocation strategies. This can provide more reliable advertising experiment data for optimizing advertising delivery strategies.

[0005] According to a first aspect of an embodiment of the present application, there is provided an advertising experiment method, which is applied to an advertising experiment system, and the method includes:

[0006] Creating multiple experimental ads according to the experimental elements of the advertising delivery experiment, wherein the experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable, and the advertising delivery strategies corresponding to the experimental ads are different, wherein one advertising delivery strategy includes a parameter group corresponding to each experimental variable;

[0007] According to a preset traffic allocation strategy, determining a first advertisement delivery strategy corresponding to a first traffic request in the advertisement delivery strategies corresponding to each experimental advertisement, wherein the first traffic request is used to trigger advertisement delivery;

[0008] A first experimental advertisement corresponding to the first advertisement delivery strategy is delivered, and experimental advertisement delivery data corresponding to the first traffic request is collected, wherein the experimental advertisement delivery data represents an advertisement effect.

[0009] Optionally, the method further includes:

[0010] Data analysis is performed on the experimental advertising delivery data corresponding to each traffic request to obtain analysis results, wherein the analysis results include advertising effects corresponding to different parameter groups of the same experimental variable and / or differences in advertising effects of different parameter groups of the same experimental variable.

[0011] Optionally, the flow distribution strategy includes a layered flow orthogonal strategy, wherein different experimental variables are located in different flow layers, and multiple parameter groups of the same experimental variable are located in the same flow layer;

[0012] According to the preset traffic allocation strategy, determining the first advertisement delivery strategy corresponding to the first traffic request in the advertisement delivery strategies corresponding to the experimental advertisements includes:

[0013] According to a preset traffic distribution strategy, determine a parameter group in each traffic layer that matches the first traffic request;

[0014] The parameter groups matching the first traffic request in each traffic layer are combined to obtain a first advertisement delivery strategy corresponding to the first traffic request.

[0015] Optionally, the method further includes: storing a mapping relationship between an advertisement delivery strategy corresponding to a historical traffic request and identification information of the historical traffic request in a mapping relationship library;

[0016] According to the preset traffic allocation strategy, determining the first advertisement delivery strategy corresponding to the first traffic request in the advertisement delivery strategies corresponding to the experimental advertisements includes:

[0017] The identification information of the first traffic request is used to search in the mapping relationship library. When a target advertising delivery strategy having a mapping relationship with the identification information of the first traffic request is stored in the mapping relationship library, the target advertising delivery strategy is determined as the first advertising delivery strategy.

[0018] Optionally, the method further includes:

[0019] Determine, according to the attribute information of the first traffic request, a first advertisement set in the current advertisement set of the advertisement experiment system, wherein the first traffic request satisfies the advertisement delivery strategy of each advertisement in the first advertisement set, and the current advertisement set includes experimental advertisements and / or non-experimental advertisements;

[0020] When the first advertisement set does not include the first experimental advertisement corresponding to the first advertisement delivery strategy, a non-experimental advertisement is selected from the non-experimental advertisements in the first advertisement set for delivery.

[0021] Optionally, the advertising experiment system stores setting parameter templates for each experimental quantity in the advertising delivery strategy;

[0022] Create multiple experimental ads based on the experimental elements of the ad delivery experiment, including:

[0023] Combining multiple parameter groups of each experimental variable in the experimental element with setting parameter templates corresponding to the experimental quantity to obtain multiple advertising delivery strategies;

[0024] Create experimental ads based on each ad delivery strategy.

[0025] Optionally, the experimental elements also include at least one of an experimental budget, an experimental cycle, and an advertising effect determination index.

[0026] Optionally, the experimental elements also include the application traffic of each parameter group in each traffic layer.

[0027] According to a second aspect of an embodiment of the present application, an advertising experiment device is provided, including:

[0028] The first unit is used to create multiple experimental advertisements according to the experimental elements of the advertisement delivery experiment, wherein the experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable, and the advertisement delivery strategies corresponding to the experimental advertisements are different, wherein one advertisement delivery strategy includes a parameter group corresponding to each experimental variable;

[0029] The second unit is used to determine, according to a preset traffic allocation strategy, a first advertisement delivery strategy corresponding to the first traffic request in the advertisement delivery strategies corresponding to each experimental advertisement, wherein the first traffic request is used to trigger the delivery of the advertisement;

[0030] The third unit is used to deliver the first experimental advertisement corresponding to the first advertisement delivery strategy, and collect experimental advertisement delivery data corresponding to the first traffic request, wherein the experimental advertisement delivery data represents an advertisement effect.

[0031] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including a memory and a processor;

[0032] The memory is connected to the processor and is used to store programs;

[0033] The processor is used to implement the advertising experiment method as described in any one of the first aspects of the embodiments of the present application by running the program in the memory.

[0034] According to a fourth aspect of the embodiments of the present application, there is provided a computer program product, comprising computer program instructions, which, when executed by a processor, enable the processor to implement the advertising experiment method as described in any one of the first aspects of the embodiments of the present application.

[0035] According to a fifth aspect of an embodiment of the present application, there is provided an advertising experiment system, including:

[0036] Server and Client, where

[0037] The client is used to obtain user input information and send the user input information to the server, wherein the user input information includes experimental elements of a first experiment, the experimental elements include at least one experimental variable and multiple parameter groups of each experimental variable, and the experimental variable includes a sub-strategy in an advertisement delivery strategy;

[0038] The server is used to implement the advertising experiment method as described in any one of the first aspects of the embodiments of the present application.

[0039] The advertising experiment method provided in the embodiment of the present application first creates multiple experimental advertisements according to the experimental elements of the advertising delivery experiment, wherein the experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable, and the advertising delivery strategies corresponding to the various experimental advertisements are different, wherein one of the advertising delivery strategies includes a parameter group corresponding to each experimental variable; then, according to a preset traffic allocation strategy, a first advertising delivery strategy corresponding to a first traffic request is determined in the advertising delivery strategies corresponding to the various experimental advertisements, wherein the first traffic request is used to trigger the delivery of the advertisement; finally, the first experimental advertisement corresponding to the first advertising delivery strategy is delivered, and experimental advertisement delivery data corresponding to the first traffic request is collected, wherein the experimental advertisement delivery data represents the advertising effect.

[0040] The advertising experiment method provided in the embodiment of the present application, on the one hand, can automatically create multiple experimental advertisements according to the experimental elements of the advertising delivery experiment, and automatically deliver advertisements according to the preset traffic allocation strategy, which significantly improves the efficiency of the experiment. At the same time, since the intervention of manual operations is reduced, the accuracy of the experiment is also improved; on the other hand, by designing a suitable traffic allocation strategy, it can ensure that each experimental advertisement can obtain relatively balanced traffic, thereby avoiding the deviation of the experimental results caused by uneven traffic distribution, and ensuring the fairness and comparability of the experimental results; on the other hand, by collecting experimental advertising delivery data, it can provide operators and advertisers with effect data of different experimental advertisements, so that they can more accurately evaluate the effects of different advertising delivery strategies, thereby optimizing advertising delivery strategies, improving advertising effects and reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0042] Figure 1 A schematic diagram of the structure of an advertising experiment system provided in an embodiment of the present application.

[0043] Figure 2 A schematic diagram of a flow chart of an advertising experiment method provided in an embodiment of the present application.

[0044] Figure 3 A schematic diagram of a flow chart of another advertising experiment method provided in an embodiment of the present application.

[0045] Figure 4 A schematic diagram of a processing flow for determining a first advertisement delivery strategy corresponding to a first traffic request provided in an embodiment of the present application.

[0046] Figure 5 A schematic diagram of another processing flow for determining a first advertisement delivery strategy corresponding to a first traffic request provided in an embodiment of the present application.

[0047] Figure 6 A schematic diagram of a flow chart of another advertising experiment method provided in an embodiment of the present application.

[0048] Figure 7 A schematic diagram of the structure of an advertising experimental device provided in an embodiment of the present application.

[0049] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solution of the embodiment of the present application is suitable for use in various scenarios where experiments are needed on advertising delivery strategies, such as Internet marketing, e-commerce promotion, mobile application promotion and other scenarios, and is widely used in digital media, marketing, brand building and other fields. In these application scenarios, there are many factors that affect the advertising effect in the advertising delivery strategy, including but not limited to the positioning of the target population, the pricing strategy of the advertisement, the selected media channel, the geographical scope of the delivery, the design and optimization of the advertising material and the landing page. In order to find the best advertising delivery strategy so that the advertisement can obtain better advertising effect, the operator needs to formulate and adjust the strategies for each key influencing factor (i.e., each sub-strategy) in the advertising delivery strategy in detail, so it is necessary to conduct advertising experiments for each sub-strategy in the advertising delivery strategy. The technical solution of the embodiment of the present application automatically creates multiple experimental advertisements, and delivers experimental advertisements based on a preset traffic allocation strategy, collects experimental advertisement delivery data that characterizes the advertising effect, can effectively improve the efficiency and accuracy of advertising experiments, reduce manual errors, and ensure the fairness of the experimental environment, so as to provide more reliable advertising experimental data for optimizing advertising delivery strategies.

[0051] The technical solution provided in the embodiments of the present application can be exemplarily applied to hardware devices such as processors, electronic devices, servers (including cloud servers), or packaged into software programs to be run. When the hardware device executes the processing of the technical solution of the embodiments of the present application, or the above-mentioned software program is run, the automatic splitting of the target task and the automatic calling of the application program interface required for the task can be achieved to complete the purpose of the target task. The embodiments of the present application only exemplarily introduce the specific processing of the technical solution of the present application, and do not limit the specific implementation form of the technical solution of the present application. Any technical implementation form that can execute the processing of the technical solution of the present application can be adopted by the embodiments of the present application.

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] Before introducing this application solution, the relevant technologies are first introduced:

[0054] In the current digital age, advertising has become an important means for companies to promote their products, services and brands. With the rapid development of Internet technology, the forms and channels of advertising have become increasingly diversified. How to effectively evaluate and optimize advertising strategies to improve advertising effectiveness and reduce costs has become a common focus for advertisers and advertising platforms.

[0055] The existing advertising experiment method usually requires operators to create multiple experimental ads, each of which contains different sets of advertising strategy parameters, so that the effect of each parameter group can be tested separately while controlling other variables. This process is extremely cumbersome, and it cannot guarantee that all ads are in a fair comparison environment throughout the process, nor can it avoid various errors caused by manual operations.

[0056] In view of this, the embodiments of the present application are committed to providing an advertising experiment method, device, equipment, storage medium, product and system, which can effectively improve the efficiency and accuracy of advertising experiments, reduce human errors, and ensure the fairness of the experimental environment by automatically creating multiple experimental advertisements and delivering experimental advertisements based on preset traffic allocation strategies. This can provide more reliable advertising experiment data for optimizing advertising delivery strategies.

[0057] Exemplary Systems

[0058] For ease of understanding, firstly, an exemplary introduction to the implementation environment of the advertising experiment method provided in the embodiment of the present application is given. Figure 1 , Figure 1 This is a schematic diagram of the structure of an advertising experiment system provided in an embodiment of the present application. The advertising experiment method provided in the present application can be exemplarily applied to the advertising experiment system.

[0059] like Figure 1 As shown, the advertising experiment system includes a client 110 and a server 120 , and the client 110 is in communication connection with the server 120 .

[0060] The client 110 is used to obtain user input information and send the user input information to the server 120 .

[0061] The user input information includes experimental elements of a first experiment, wherein the experimental elements include at least one experimental variable and a plurality of parameter groups of each experimental variable, and the experimental variable includes a sub-strategy in an advertisement delivery strategy.

[0062] The server 120 is used to execute the advertising experiment method provided by the embodiment of the present application after receiving the experimental elements of the advertising delivery experiment sent by the client 110, including: creating multiple experimental advertisements according to the experimental elements of the advertising delivery experiment, the experimental elements including at least one experimental variable and multiple parameter groups for each experimental variable, and the advertising delivery strategies corresponding to each experimental advertisement are different, wherein one of the advertising delivery strategies includes a parameter group corresponding to each experimental variable; according to a preset traffic allocation strategy, determining a first advertising delivery strategy corresponding to a first traffic request in the advertising delivery strategies corresponding to each experimental advertisement, the first traffic request being used to trigger the delivery of an advertisement; delivering the first experimental advertisement corresponding to the first advertising delivery strategy, and collecting experimental advertisement delivery data corresponding to the first traffic request, the experimental advertisement delivery data representing the advertising effect.

[0063] As for the specific content of the advertising experimental method, it will be explained in the subsequent embodiments. Please refer to the content of the subsequent embodiments and will not be explained in detail here.

[0064] The client 110 can be various terminal devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc., and can also be applications running on the terminal device, such as web browsers, desktop applications, mobile applications, etc., which are not limited in this application.

[0065] The server 120 may be an independent physical server, or a server cluster consisting of multiple physical servers, or a cloud server or a virtual machine providing cloud computing services, which is not limited in this application.

[0066] The communication network may be any form of wireless communication network or wired communication network, or any combination thereof, which is not limited in this application.

[0067] Exemplary Methods

[0068] Figure 2 The following is a flow chart of an advertising experiment method provided in an embodiment of the present application. Figure 2 As shown, the advertising experimental method can be applied to Figure 1 The advertising experiment system in the advertising experiment system is specifically executed by a server in the advertising experiment system. The method includes steps S201-S203:

[0069] S201. Create multiple experimental advertisements according to the experimental elements of the advertising delivery experiment, wherein the experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable, and the advertising delivery strategies corresponding to the various experimental advertisements are different, wherein one advertising delivery strategy includes a parameter group corresponding to each experimental variable.

[0070] Advertising strategy can be understood as a series of plans and methods adopted in advertising activities to achieve marketing goals, aiming to maximize advertising effects, enable advertising budgets to accurately hit targets, and optimize cost efficiency.

[0071] Advertising strategies usually include multiple dimensions, including but not limited to advertising materials (including pictures, videos, copywriting, etc.), landing page design (the page that users jump to after clicking on the ad), delivery time (the specific time period for ad display), media selection (platform or channel for ad delivery), target population (segmented groups of ad audiences), RTA (Real-Time Advertising, real-time advertising optimization) strategy, JS template (JavaScript template used to dynamically generate advertising content), etc. Each dimension can be regarded as a sub-strategy of the advertising delivery strategy, and a complete advertising delivery strategy includes multiple sub-strategies. Different settings or combinations of each sub-strategy of the advertising delivery strategy will have an impact on the advertising effect, so it is a key research object in advertising delivery experiments.

[0072] The experimental variables can be understood as factors that can be changed in the advertising experiment and whose changes will affect the advertising effect. By changing these factors, their specific impact on the advertising effect can be observed and analyzed.

[0073] Optionally, an experimental variable corresponds to an adjustable sub-strategy in the advertising strategy. For example, in an advertising experiment, the experimental variable can correspond to different designs of advertising materials, different versions of landing pages, differences in delivery time, selection of target groups, allocation of media channels, adjustment of real-time bidding (RTA) strategies, changes in JavaScript (JS) templates, etc.

[0074] The parameter groups of the experimental variables can be understood as different possible states or settings of the experimental variables in the advertising experiment. By changing the parameter values ​​of the experimental variables, the specific impact of the experimental variables on the advertising effect can be observed and analyzed. Each experimental variable corresponds to multiple parameter groups. For example, if the experimental variable is the advertising material, the parameter group may include different pictures, videos or animation designs; if the experimental variable is the delivery time, the parameter group may include different time periods or dates; if the experimental variable is the target audience, the parameter group may include different audience characteristics such as age, gender, interests and hobbies.

[0075] The experimental elements can be obtained in a variety of ways, depending on the needs of advertisers, available data resources, and the functions of the advertising experiment system, etc., which are not limited in this application. Optionally, the system can follow the design principles of A / B testing or multivariate testing to select experimental elements, including ensuring the independence of experimental variables, controlling other potential influencing factors, and setting up appropriate control groups.

[0076] Optionally, the experimental elements can be obtained by the user through the client input. The advertising experiment system usually provides a user-friendly interface, which facilitates the user to manually input the experimental elements through the client interface of the advertising experiment system, including selecting or defining experimental variables and setting different parameter groups for each experimental variable.

[0077] Optionally, the advertising experiment system can analyze historical advertising campaign data, identify key factors that may have a significant impact on advertising effectiveness, determine these factors as experimental variables, and set parameter groups for each experimental variable based on historical performance.

[0078] Optionally, the ad experiment system can use machine learning algorithms to automatically identify and select experimental variables. The algorithm analyzes large amounts of ad data to identify key factors associated with ad performance and recommends these factors as experimental variables. In addition, the algorithm can also predict the potential effects of different parameter groups based on data patterns, thereby optimizing the experimental design.

[0079] After obtaining the experimental elements of the advertising experiment, you can create multiple experimental ads based on the experimental elements of the advertising experiment.

[0080] Based on the selected experimental variables and parameter groups, multiple experimental ads can be created. Each experimental ad corresponds to a unique ad delivery strategy, which includes a parameter group for each experimental variable. For example, if the experimental variables include creative and delivery time, and each variable has two parameter groups, there will be a total of 4 (2x2) different ad delivery strategies, that is, 4 experimental ads, each of which will have a specific creative version and a specific delivery time period.

[0081] As an optional implementation, the advertising experiment system stores setting parameter templates for each experimental quantity in the advertising delivery strategy.

[0082] In this implementation, multiple experimental ads are created according to the experimental elements of the advertising delivery experiment, including steps A1-A2:

[0083] A1. Combining multiple parameter groups of each experimental variable in the experimental element with setting parameter templates corresponding to the experimental quantity to obtain multiple advertising delivery strategies.

[0084] Specifically, the advertising experiment system stores setting parameter templates for each sub-strategy in the advertising delivery strategy. The setting parameter templates are pre-defined and include setting parameters for each sub-strategy under common or recommended circumstances. Optionally, the setting parameter templates can be summarized based on historical data, industry best practices, or advertisers' own experience.

[0085] The experimental quantitative corresponding advertising delivery strategy refers to other types of sub-strategies except the sub-strategies corresponding to the experimental variables. The setting parameters of these sub-strategies are usually relatively stable and will not change frequently, so they can be managed in a templated manner.

[0086] Optionally, the experimental quantitative corresponding advertisement delivery strategy may correspond to other types of sub-strategies except for the sub-strategies corresponding to the experimental variables.

[0087] After obtaining the experimental elements and determining the experimental variables, the experimental quantity can be determined according to the experimental variables, and then the setting parameter templates corresponding to the experimental quantity can be retrieved from the advertising experiment system. These templates provide the basic framework and recommended values ​​for the experimental quantity. Of course, users can also modify the setting parameter templates of the experimental quantity according to actual needs to meet specific experimental purposes.

[0088] After obtaining the setting parameter template corresponding to the experimental quantity, the system automatically combines multiple parameter groups of each experimental variable in the experimental element with the setting parameter template corresponding to the experimental quantity. In this way, multiple advertising delivery strategies with differences can be quickly generated, and each combination forms a complete advertising delivery strategy, and each advertising delivery strategy includes a parameter group for each experimental variable and a setting parameter template for the experimental quantity.

[0089] A2. Create experimental ads based on each advertising strategy.

[0090] After obtaining multiple advertising delivery strategies, experimental ads will be created according to each advertising delivery strategy.

[0091] Optionally, create a new ad instance for each strategy in the ad experiment system, and configure the various attributes of the ad according to the setting parameters in the strategy. For example, according to the ad creative requirements in the ad delivery strategy, create or select corresponding ad content, such as pictures, videos, copywriting, etc.; for example, configure corresponding delivery settings for each experimental ad, including the ad budget, bidding method, delivery channel, target audience, etc.

[0092] Optionally, preview and test each experimental ad before it is officially launched to ensure that the ad content, format, and delivery settings meet the advertiser's requirements and expectations. If any problems are found or adjustments are needed, make modifications and optimizations in a timely manner.

[0093] In this implementation, by storing and using parameter setting templates, multiple differentiated advertising delivery strategies can be quickly generated without manually setting each parameter one by one. This not only reduces the time and error risk of manual configuration, but also improves the diversity and flexibility of advertising delivery strategies, greatly improving the efficiency and accuracy of creating experimental ads. In addition, the recommended values ​​in the template are based on historical data or best practices, which helps reduce blindness and trial and error costs in experiments.

[0094] S202. According to a preset traffic allocation strategy, determine a first advertisement delivery strategy corresponding to a first traffic request in the advertisement delivery strategies corresponding to each experimental advertisement, where the first traffic request is used to trigger advertisement delivery.

[0095] The traffic allocation strategy can be understood as the rules or methods used to determine how to allocate traffic to different advertising strategies in an advertising experiment. It ensures the fairness and effectiveness of the experiment. By reasonably allocating traffic, the experimental ads corresponding to each advertising strategy can have the opportunity to obtain sufficient exposure and testing.

[0096] The traffic request can be understood as a request made by the system to display an ad when a user visits a webpage or uses an application. During the ad experiment, the traffic request is the trigger point for ad delivery. Whenever a user's behavior triggers the conditions for ad display (such as opening a new page, scrolling to a specific location, clicking a button, etc.), the ad experiment system will receive a traffic request, which will then trigger the ad delivery corresponding to the traffic request.

[0097] After receiving the first traffic request, the advertising experiment system will conduct a comprehensive analysis based on the first traffic request and the advertising delivery strategies corresponding to each experimental advertisement, so as to determine which advertising delivery strategy the first traffic request should be assigned to and which advertising delivery strategy should be used to deliver the corresponding experimental advertisement according to the preset traffic allocation strategy.

[0098] Optionally, the traffic allocation strategy may include multi-dimensional content such as user feature matching, experimental condition judgment, random or deterministic selection, etc. Exemplarily, the system may analyze the user characteristics corresponding to the first traffic request, such as geographic location, age, gender, interest preferences, etc., in order to find the advertising delivery strategy that best matches the user. Exemplarily, based on the experimental design, the system will determine whether the current traffic request meets the conditions of a specific experiment (such as a specific time window, a specific page visit, etc.). Exemplarily, based on the specific design of the traffic allocation strategy, the system may use random selection or deterministic selection to determine the first advertising delivery strategy, where random selection is usually used to ensure the randomness and fairness of the experiment, while deterministic selection may be based on specific rules or algorithms.

[0099] As an optional implementation method, the traffic distribution strategy includes at least one of a layered traffic orthogonal strategy, a same-layer traffic mutual exclusion strategy, and a traffic recovery strategy, which will be explained in subsequent content and will not be elaborated here.

[0100] After comprehensive analysis and evaluation based on the traffic allocation strategy, the system will select an advertising delivery strategy that best matches the first traffic request as the first advertising delivery strategy. This strategy will determine which experimental ad to display to the user who triggered the first traffic request.

[0101] Optionally, after determining the first advertising delivery strategy, the system will record the decision-making process and results so that when analyzing the experimental results later, it can trace how each traffic request is allocated, which helps to ensure the transparency and repeatability of the experiment.

[0102] S203: Place a first experimental advertisement corresponding to the first advertisement placement strategy, and collect experimental advertisement placement data corresponding to the first traffic request, wherein the experimental advertisement placement data represents an advertisement effect.

[0103] After the first advertisement delivery strategy is determined, the system delivers the first experimental advertisement corresponding to the first advertisement delivery strategy.

[0104] Optionally, delivering the first experimental advertisement can be understood as sending the first experimental advertisement to the media that generates the first traffic request. After delivering the first experimental advertisement, it will automatically participate in the media's advertising bidding according to the bidding strategy in the first advertising delivery strategy. If the bidding is successful, that is, the bid is higher than other competitors and meets the display requirements of the first traffic request, then the first experimental advertisement will be displayed to the user who generated the first traffic request.

[0105] After the first experimental ad is launched, the system will collect data related to the advertising effect in real time. This data may include but is not limited to key indicators such as the number of ad exposures, clicks, click-through rates, and conversion rates. The collected experimental ad launch data will be used for subsequent advertising effect analysis and evaluation. By comparing the experimental effects under different advertising strategies, it is possible to identify which strategies are more effective and which strategies need to be improved. At the same time, data analysis can also help understand the behavioral habits and preferences of the target audience, providing strong support for future advertising strategies.

[0106] Optionally, when the experimental elements of the advertising delivery experiment include an experimental cycle and / or monitoring indicators, the advertising delivery effect is monitored in real time and data is collected according to the preset experimental cycle and monitoring indicators.

[0107] The advertising experiment method provided in the embodiment of the present application first creates multiple experimental advertisements according to the experimental elements of the advertising delivery experiment, wherein the experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable, and the advertising delivery strategies corresponding to the various experimental advertisements are different, wherein one of the advertising delivery strategies includes a parameter group corresponding to each experimental variable; then, according to a preset traffic allocation strategy, a first advertising delivery strategy corresponding to a first traffic request is determined in the advertising delivery strategies corresponding to the various experimental advertisements, wherein the first traffic request is used to trigger the delivery of the advertisement; finally, the first experimental advertisement corresponding to the first advertising delivery strategy is delivered, and experimental advertisement delivery data corresponding to the first traffic request is collected, wherein the experimental advertisement delivery data represents the advertising effect.

[0108] The advertising experiment method provided in the embodiment of the present application, on the one hand, can automatically create multiple experimental advertisements according to the experimental elements of the advertising delivery experiment, and automatically deliver advertisements according to the preset traffic allocation strategy, which significantly improves the efficiency of the experiment. At the same time, since the intervention of manual operations is reduced, the accuracy of the experiment is also improved; on the other hand, by designing a suitable traffic allocation strategy, it can ensure that each experimental advertisement can obtain relatively balanced traffic, thereby avoiding the deviation of the experimental results caused by uneven traffic distribution, and ensuring the fairness and comparability of the experimental results; on the other hand, by collecting experimental advertising delivery data, it can provide operators and advertisers with effect data of different experimental advertisements, so that they can more accurately evaluate the effects of different advertising delivery strategies, thereby optimizing advertising delivery strategies, improving advertising effects and reducing costs.

[0109] As an optional implementation, Figure 3 As shown, the method further includes step S301:

[0110] S301. Perform data analysis on the experimental advertising delivery data corresponding to each traffic request to obtain analysis results, wherein the analysis results include advertising effects corresponding to different parameter groups of the same experimental variable and / or differences in advertising effects of different parameter groups of the same experimental variable.

[0111] The object of data analysis is the experimental advertising delivery data corresponding to each traffic request. According to the above content, the experimental advertising delivery data can represent the advertising effect, including but not limited to the number of advertising exposures, number of clicks, click-through rate, conversion rate and other data.

[0112] The main goal of data analysis is to evaluate the advertising effectiveness of different parameter groups for the same experimental variable and / or the differences in advertising effectiveness between these parameter groups.

[0113] The analysis results of data analysis will provide detailed insights into the effects of different parameter groups. For example, it may be found that ads with a certain color have a higher click-through rate, while ads with another color have a higher conversion rate. The analysis results will serve as an important basis for subsequent advertising optimization strategies. For example, the color, layout, copywriting, etc. of the advertisement can be adjusted according to the analysis results to improve the advertising effect.

[0114] Optionally, when the experimental elements of an advertising placement experiment include an advertising effect determination index, data analysis is performed based on the advertising effect determination index as a core indicator, and the analysis results use relevant data analysis results (core indicator values, relative difference values, confidence intervals, etc.) of the advertising effect determination index for different experimental advertisements to characterize the advertising effects corresponding to different parameter groups of the same experimental variable and / or the differences in advertising effects for different parameter groups of the same experimental variable.

[0115] Optionally, the analysis results of the data analysis also include the final experimental conclusion determined based on the advertising effects corresponding to different parameter groups of the same experimental variable and / or the difference in advertising effects of different parameter groups of the same experimental variable. Exemplarily, the experimental conclusion is "statistically significant, experimental version 2 is the winning version, and it is recommended to launch this configuration to improve core indicators."

[0116] Optionally, you can use data visualization tools to intuitively display the analysis results, such as bar charts, line charts, scatter plots, etc.

[0117] Specifically, after determining the analysis object and analysis target, determine the appropriate data analysis method according to the actual scenario and needs, perform data analysis on the experimental advertising data corresponding to each traffic request, and obtain the analysis results. Based on the results of the data analysis, we can explain the advertising effects corresponding to different parameter groups of the same experimental variable, as well as the differences in advertising effects between these parameter groups, which helps to understand which parameter groups perform better in advertising delivery and why such differences occur.

[0118] The data analysis method is determined according to the actual application scenario and requirements, and this application does not limit this.

[0119] By dividing the traffic according to the traffic allocation strategy in step S202, the experimental ads corresponding to each advertising delivery strategy will obtain the same traffic. However, due to the differences in parameters of the advertising delivery strategy itself (such as different bidding sub-strategies, etc.), each experimental ad may not necessarily obtain the same exposure. To avoid this difference in sample quantity, as an optional implementation method, the data analysis method may adopt the Welch's t-test evaluation method.

[0120] Optionally, when analyzing data using the Welch's t-test evaluation method, follow the steps below to ensure the accuracy and reliability of the results.

[0121] First, based on the experimental advertising data, the mean and variance of each parameter group (such as ads with different colors, layouts or copywriting) and the number of samples are calculated. These data are the basis for evaluating the difference in advertising effects. Next, the calculated mean, variance and sample size are used to calculate the t distribution, degrees of freedom and standard error. Finally, based on the selection of an appropriate confidence level (such as 95%) for the t distribution, the corresponding critical value is found by looking up the table, and then the confidence interval is calculated.

[0122] The calculation formula of t distribution is:

[0123]

[0124] The degree of freedom calculation formula is:

[0125]

[0126] The standard error calculation formula is:

[0127]

[0128] The confidence interval calculation formula is:

[0129]

[0130] Among them, m A 、m B is the mean, is the variance, n A 、n B is the sample size.

[0131] Finally, we can determine the conclusion of the experiment based on the evaluation results of Welch's t-test. For example, if the advertising effect of experimental version 2 is statistically significantly better than version 1, we can conclude: "The statistics are significant, experimental version 2 is the winning version, and it is recommended to launch this configuration to improve core indicators." Such a conclusion will provide strong data support for the formulation of advertising strategies. In addition, in order to display the analysis results more intuitively, we can also use data visualization tools such as bar charts and line charts to graphically present the differences in advertising effects of different parameter groups, helping decision makers to understand the meaning behind the data more quickly.

[0132] In the field of advertising, evaluation indicators can be roughly divided into the following types:

[0133] For normal distribution indicators, such as CPM (cost per thousand impressions), you can directly calculate the corresponding mean and variance and substitute them into the formula.

[0134] Binomial distribution indicators, such as click-through rate and conversion rate, can be represented by 0 or 1 for each sample, which naturally conforms to the binomial distribution. Therefore, the mean and variance of the sample can be calculated very quickly. Assume that the click-through rate of sample A is P A , then its m A =P A ,and

[0135] Skewed indicators, such as conversion cost indicators, do not conform to the normal distribution when they fall on each exposure, and are very likely to fall on high-priced traffic, traffic from a specific group, etc.; secondly, most exposures do not lead to conversions, and it is impossible to define the conversion cost. According to statistical rules, we use the method of extracting samples, such as extracting 5,000 exposures as a sample data. Such sample data still conforms to the normal distribution, and the significance calculation is performed by repeatedly extracting a sample set.

[0136] The experimental system completes a rigorous analysis of the experimental group data based on statistical rules, allowing the executors to directly and intuitively see whether the experimental group effect has been significantly improved or reduced.

[0137] As an optional implementation, the experimental elements also include at least one of an experimental budget, an experimental cycle, and an advertising effect determination index.

[0138] The experiment budget can be understood as the total amount of funds allocated for the advertising experiment. It determines the scale, duration and intensity of the experiment.

[0139] The experimental cycle can be understood as the time period from the beginning to the end of the advertising experiment. A reasonable experimental cycle can ensure that the experiment has enough time to collect data, analyze the effect and optimize, so as to draw reliable experimental conclusions.

[0140] Advertising effectiveness indicators are standards or criteria used to measure the effectiveness of advertising experiments. They can help advertisers understand the performance of advertisements among the target audience and whether the expected goals have been achieved. Common advertising effectiveness indicators include click-through rate (CTR), conversion rate (CVR), exposure, cost-effectiveness ratio (ROI), etc. These indicators can be selected and combined according to the purpose of the experiment and the needs of advertisers to comprehensively and accurately evaluate the advertising effectiveness.

[0141] In advertising experiments, the experimental budget, experimental cycle and advertising effect judgment indicators are interrelated and influence each other. The experimental budget determines the scale and duration of the experiment, which in turn affects the collection and analysis of experimental data; the length of the experimental cycle directly affects the adequacy and reliability of the experimental data; and the advertising effect judgment indicators are an important basis for measuring the experimental effect. Their selection and setting need to be carefully considered based on the purpose of the experiment and the needs of advertisers. By reasonably setting these experimental elements, the effective implementation of advertising experiments can be ensured, and valuable references and bases can be provided for advertisers.

[0142] As an optional implementation, the traffic allocation strategy includes a layered traffic orthogonal strategy, wherein different experimental variables are located in different traffic layers, and multiple parameter groups of the same experimental variable are located in the same traffic layer.

[0143] like Figure 4 As shown, according to the preset traffic allocation strategy, in the advertising delivery strategies corresponding to the experimental advertisements, a first advertising delivery strategy corresponding to the first traffic request is determined, including steps S401-S402:

[0144] S401. Determine a parameter group in each traffic layer that matches the first traffic request according to a preset traffic allocation strategy.

[0145] In step S201, multiple experimental ads have been created based on the experimental elements of the advertising delivery experiment, and these experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable. Each experimental ad corresponds to a unique advertising delivery strategy, which is composed of a parameter group corresponding to each experimental variable.

[0146] In step S202, it is necessary to determine the first advertisement delivery strategy corresponding to the first traffic request according to a preset traffic distribution strategy. This implementation provides an optional traffic distribution strategy, namely, a layered traffic orthogonal strategy.

[0147] In this strategy, different experimental variables are assigned to different traffic layers, and each traffic layer corresponds to an experimental variable. For example, if there are two experimental variables in the experiment (such as material and bidding), then these two variables will be placed in two different traffic layers. In addition, multiple parameter groups of the same experimental variable are located in the same traffic layer, and these parameter groups are orthogonal in traffic distribution, that is, each parameter group has the opportunity to be independently assigned to the traffic without being affected by other parameter groups. This strategy ensures that each traffic request will be matched with a parameter group of each experimental variable when passing through each traffic layer, thereby realizing the orthogonal design of the advertising delivery strategy.

[0148] Specifically, in step S401, for each traffic layer, we have a preset traffic allocation rule, which determines which parameter group will be allocated to the current traffic request. When the first traffic request arrives, the system needs to traverse each traffic layer in turn, and determine a parameter group that matches the first traffic request in each traffic layer according to the preset traffic allocation rule. Finally, the parameter group that matches the first traffic request in each traffic layer will be determined.

[0149] As an optional implementation method, a hash algorithm is used to determine a parameter group that matches the first traffic request in each traffic layer.

[0150] A hash algorithm is an algorithm that maps input data (such as strings, numbers, etc.) to an output value of a fixed size (i.e., a hash value). In the scenario of traffic distribution, we can use the characteristics of the hash algorithm to take certain attributes of the first traffic request (such as user ID, session ID, timestamp, etc.) as input to generate a hash value, and then determine which parameter group to match based on this hash value.

[0151] First, you need to select a suitable hash function, which should have good hash performance, that is, it should be able to map different inputs to different hash values ​​as much as possible to reduce the possibility of hash conflicts. Secondly, for each traffic layer, extract the relevant attributes of the first traffic request, and use these attributes as inputs to the hash function to calculate the corresponding hash value. Next, map the calculated hash value to the preset parameter group, which can be achieved through modulo operation or lookup table. If necessary, the mapping relationship between the identification information of the first traffic request and the determined parameter group can also be stored in the mapping relationship library for subsequent use or analysis.

[0152] In this implementation, on the one hand, the computational complexity of the hash algorithm is low and it can quickly process a large number of traffic requests. On the other hand, by reasonably designing the hash function and mapping strategy, it can ensure that the traffic is evenly distributed among the various parameter groups, thereby avoiding the situation where some parameter groups are overloaded while other parameter groups are idle. On the other hand, when the number of experimental variables or parameter groups increases, it is only necessary to adjust the hash function and mapping strategy accordingly without making major changes to the entire traffic distribution model.

[0153] S402: Combine the parameter groups in each traffic layer that match the first traffic request to obtain a first advertisement delivery strategy corresponding to the first traffic request.

[0154] After determining the parameter groups that match the first traffic request in each traffic layer, the system combines these parameter groups to form a complete advertising delivery strategy, which is the first advertising delivery strategy corresponding to the first traffic request.

[0155] The first advertising delivery strategy determines which experimental advertisement should be delivered to the current traffic request under the first traffic request, as well as various delivery parameters of the experimental advertisement (such as advertising copy, bidding strategy, targeting conditions, etc.).

[0156] This optional implementation achieves refined control over the ad delivery strategy through a layered traffic orthogonal strategy. It ensures that each traffic request can be matched with the parameter group of each experimental variable, resulting in a comprehensive and orthogonal ad delivery strategy. This strategy helps to more accurately evaluate the impact of different parameter groups on ad performance, providing strong support for subsequent ad optimization. The layered traffic orthogonal strategy ensures that each experimental variable and parameter group has the opportunity to be evaluated independently, so as to more accurately understand their specific contribution to ad performance. This approach helps us optimize ads more effectively and improve ad delivery results.

[0157] As an optional implementation, the experimental elements also include the application traffic of each parameter group in each traffic layer.

[0158] Application traffic can be understood as the proportion or amount of traffic that each parameter group expects to obtain in the advertising experiment. It reflects the experimenter's expectations and preferences for the effect of each parameter group and is also one of the important bases for the traffic allocation strategy. In the layered traffic orthogonal strategy, application traffic is an important part of the traffic allocation strategy. The parameter groups in each traffic layer will obtain the corresponding traffic proportion according to their application traffic. By reasonably setting the application traffic and adopting an effective traffic allocation strategy (such as hashing algorithm), it can be ensured that each parameter group has the opportunity to be evaluated independently, so as to more accurately understand their specific contribution to the advertising effect.

[0159] As an optional implementation manner, the method further includes: storing a mapping relationship between an advertisement delivery strategy corresponding to a historical traffic request and identification information of the historical traffic request in a mapping relationship library.

[0160] like Figure 5 As shown, according to the preset traffic allocation strategy, in the advertisement delivery strategies corresponding to the experimental advertisements, a first advertisement delivery strategy corresponding to the first traffic request is determined, including step S501:

[0161] S501. Use the identification information of the first traffic request to search in the mapping relationship library. When a target advertising delivery strategy that has a mapping relationship with the identification information of the first traffic request is stored in the mapping relationship library, determine the target advertising delivery strategy as the first advertising delivery strategy.

[0162] A mapping relationship library is pre-established to store the mapping relationship between historical traffic requests and corresponding advertising delivery strategies. This library can be a database, a cache system, or any data structure that can efficiently store and retrieve data.

[0163] Whenever a traffic request comes in, the system will store the identification information of the traffic request (such as user ID, session ID, timestamp, etc.) and the corresponding advertising delivery strategy (including a parameter group for each experimental variable) in the mapping relationship library. As time goes by, new traffic requests and corresponding delivery strategies will continue to be added to the mapping relationship library.

[0164] When the first traffic request arrives, the system first extracts the identification information of the first traffic request and uses this information to search in the mapping relationship library. In the mapping relationship library, the system will check whether there is a target advertising delivery strategy that has a mapping relationship with the identification information of the first traffic request. If such a target advertising delivery strategy exists, the system will directly determine it as the first advertising delivery strategy and deliver the advertisement.

[0165] If there is no mapping relationship, the system may need to dynamically determine the first advertising delivery strategy based on a preset traffic allocation strategy (such as a layered traffic orthogonal strategy), and store it together with the identification information of the first traffic request in the mapping relationship library for subsequent use.

[0166] In this implementation, on the one hand, by storing and retrieving mapping relationships, the system can more quickly determine the delivery strategy corresponding to the traffic request, thereby improving the efficiency of advertising delivery; on the other hand, for the same traffic request, the system always returns the same delivery strategy, ensuring the consistency and stability of advertising delivery; on the other hand, the stored mapping relationship can be used as a historical record for subsequent data analysis, troubleshooting and auditing.

[0167] In the AB testing scenario in the field of advertising, since the bidding requirements for traffic are strict and traffic resources are limited, it is necessary to use every piece of traffic efficiently. When a piece of traffic does not meet the requirements of experimental ads A or B, in order to avoid wasting traffic, the system will use other non-experimental ads to fill this traffic. This process is called traffic recycling.

[0168] In the process of programmatic advertising, the calculation time window for ad recall and filling is very short. If the system tries to recall the experimental ad first, and then recalls the non-experimental ad for filling after the recall fails, the entire processing process may fail due to timeout, resulting in wasted traffic.

[0169] To avoid this situation, this application proposes another optional advertising experiment method, such as Figure 6As shown, the method further includes steps S601-S602:

[0170] S601. According to the attribute information of the first traffic request, determine a first advertisement set in the current advertisement set of the advertising experiment system, the first traffic request satisfies the advertisement delivery strategy of each advertisement in the first advertisement set, and the current advertisement set includes experimental advertisements and / or non-experimental advertisements.

[0171] The system first determines a first advertisement set in the current advertisement set of the advertisement experiment system according to the attribute information of the first traffic request. The first advertisement set includes all advertisements that meet the advertisement delivery strategy of the first traffic request, whether they are experimental advertisements or non-experimental advertisements.

[0172] Among them, the attribute information of the first traffic request can be understood as a series of characteristics and parameters related to the traffic request, which are usually used to describe the characteristics, sources, user behaviors and possible delivery targets of the traffic request. The attribute information of the first traffic request may include but is not limited to user information (such as user ID, user portrait, user device information, geographic location, etc.), request information (such as request time, request source, request type, etc.), context information (such as current page information, browsing history, previous behavior, etc.), and delivery targets (such as advertising space information, bidding requirements, etc.).

[0173] The current ad set is a set of all available ads in the ad experiment system, including experimental ads that are being tested (such as version A and version B in AB testing) and regular non-experimental ads, which may target different user groups and ad delivery strategies. Experimental ads are usually versions of ads that are being tested, used to compare the effects of different ad strategies. Non-experimental ads are regularly delivered ads that are not used for experimental comparisons. In the first ad set, both experimental ads and non-experimental ads may be included, depending on the attribute information of the traffic request and the ad delivery strategy.

[0174] The first advertisement set includes all advertisements that may be suitable for the current traffic request. The screening process usually involves matching the attribute information of the traffic request with the advertisement delivery strategy.

[0175] S602: When the first advertisement set does not include the first experimental advertisement corresponding to the first advertisement delivery strategy, select a non-experimental advertisement from the non-experimental advertisements in the first advertisement set for delivery.

[0176] The system checks whether the first advertisement set includes a first experimental advertisement corresponding to the first advertisement delivery strategy.

[0177] If the first ad set contains the first experimental ad, the system will select this ad for delivery.

[0178] If the first advertising set does not include the first experimental advertising corresponding to the first advertising delivery strategy, it means that the attribute information of the current traffic request does not match the delivery strategy of the first experimental advertising, or the first experimental advertising is currently unavailable for some reason (such as budget limitation, delivery time limitation, etc.), then the system will select one from the non-experimental advertising in the first advertising set for delivery.

[0179] Non-experimental advertisements are selected from the non-experimental advertisements in the first advertisement set for delivery. This selection process is usually based on a series of algorithms and strategies, aiming to maximize advertising effects (such as click-through rate, conversion rate, etc.) and user experience. For example, the system may consider multiple factors, such as the relevance of advertisements, user interest preferences, and bidding for advertisements. By comprehensively considering these factors, the system can display advertisements that best meet the needs and interests of users.

[0180] Step S602 ensures that when the first set of advertisements does not contain the first experimental advertisement, the system can quickly and accurately select one of the remaining non-experimental advertisements for delivery. This mechanism not only ensures the effective use of traffic, but also ensures the continuity and consistency of user experience. At the same time, by continuously optimizing the delivery strategy of non-experimental advertisements, the advertising effect and the return on investment of advertisers can be further improved.

[0181] This implementation, on the one hand, by considering both experimental and non-experimental ads, the system can complete the recall and filling of ads within a time window, avoiding traffic waste due to timeouts; on the other hand, it can flexibly select the most appropriate ads for delivery based on the attribute information of traffic requests and the actual situation of the current ad set. Since the system can find the most appropriate ad for each traffic in a short time, it can improve the user's ad experience and reduce user loss due to irrelevant ads or slow loading.

[0182] It should be noted that the advertising experiment system can have multiple advertising experiments at the same time. Optionally, considering that different experiments cannot be cross-tested, the concept of traffic domain is introduced.

[0183] As an optional implementation method, a unique traffic domain is allocated to each advertising experiment, and the traffic domains are mutually exclusive. The entire traffic will be divided into several experimental traffic domains and one public traffic domain. The public traffic domain will be directly used to recall non-experimental ads. Each traffic domain is divided into multiple traffic layers according to the experimental variables of the corresponding advertising experiment. Each traffic layer corresponds to an experimental variable of the traffic delivery experiment, and each traffic bucket in the traffic layer corresponds to a parameter group of the experimental variable. For each parameter group in each traffic layer in each traffic domain, a traffic mutually exclusive allocation strategy is adopted between parameter groups. For parameter groups of different experimental variables of the same advertising delivery experiment, a layered traffic orthogonal allocation strategy is adopted. Specifically, for each traffic request, the first hash operation is performed according to the device number identification information to determine the target traffic domain of this traffic, and then the second hash operation is performed according to the device number identification information to determine the target traffic bucket of the second traffic layer of the target traffic domain where this traffic falls; continue to perform this step until the target traffic bucket of each traffic layer in the target traffic domain is determined.

[0184] Exemplary Devices

[0185] Corresponding to the above-mentioned advertising experiment method, the embodiment of the present application also provides an advertising experiment device. Figure 7 is a schematic diagram of the structure of an advertising experimental device provided in an embodiment of the present application, such as Figure 7 As shown, the advertising experimental device provided in the embodiment of the present application includes:

[0186] The first unit 701 is used to create multiple experimental advertisements according to the experimental elements of the advertisement delivery experiment, wherein the experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable, and the advertisement delivery strategies corresponding to the experimental advertisements are different, wherein one advertisement delivery strategy includes a parameter group corresponding to each experimental variable;

[0187] The second unit 702 is used to determine, according to a preset traffic allocation strategy, a first advertisement delivery strategy corresponding to a first traffic request in the advertisement delivery strategies corresponding to each experimental advertisement, wherein the first traffic request is used to trigger advertisement delivery;

[0188] The third unit 703 is used to deliver the first experimental advertisement corresponding to the first advertisement delivery strategy, and collect experimental advertisement delivery data corresponding to the first traffic request, wherein the experimental advertisement delivery data represents an advertisement effect.

[0189] The advertising experiment device provided in the embodiment of the present application, on the one hand, can automatically create multiple experimental advertisements according to the experimental elements of the advertising delivery experiment, and automatically deliver advertisements according to the preset traffic allocation strategy, which significantly improves the efficiency of the experiment. At the same time, since the intervention of manual operations is reduced, the accuracy of the experiment is also improved; on the other hand, by designing a suitable traffic allocation strategy, it can ensure that each experimental advertisement can obtain relatively balanced traffic, thereby avoiding the deviation of the experimental results caused by uneven traffic distribution, and ensuring the fairness and comparability of the experimental results; on the other hand, by collecting experimental advertising delivery data, it can provide operators and advertisers with effect data of different experimental advertisements, so that they can more accurately evaluate the effects of different advertising delivery strategies, thereby optimizing advertising delivery strategies, improving advertising effects and reducing costs.

[0190] Optionally, the device further comprises:

[0191] The fourth unit is used to perform data analysis on the experimental advertising delivery data corresponding to each traffic request to obtain analysis results, wherein the analysis results include advertising effects corresponding to different parameter groups of the same experimental variable and / or differences in advertising effects of different parameter groups of the same experimental variable.

[0192] Optionally, the flow distribution strategy includes a layered flow orthogonal strategy, wherein different experimental variables are located in different flow layers, and multiple parameter groups of the same experimental variable are located in the same flow layer;

[0193] The second unit 702 can be specifically used for:

[0194] According to a preset traffic distribution strategy, determine a parameter group in each traffic layer that matches the first traffic request;

[0195] The parameter groups matching the first traffic request in each traffic layer are combined to obtain a first advertisement delivery strategy corresponding to the first traffic request.

[0196] Optionally, the device further comprises:

[0197] A fifth unit, configured to store a mapping relationship between an advertisement delivery strategy corresponding to a historical traffic request and identification information of the historical traffic request in a mapping relationship library;

[0198] The second unit 702 can be specifically used for:

[0199] The identification information of the first traffic request is used to search in the mapping relationship library. When a target advertising delivery strategy having a mapping relationship with the identification information of the first traffic request is stored in the mapping relationship library, the target advertising delivery strategy is determined as the first advertising delivery strategy.

[0200] Optionally, the device further comprises:

[0201] The sixth unit is used to determine a first advertisement set in the current advertisement set of the advertising experiment system according to the attribute information of the first traffic request, the first traffic request satisfies the advertisement delivery strategy of each advertisement in the first advertisement set, and the current advertisement set includes experimental advertisements and / or non-experimental advertisements; when the first experimental advertisement corresponding to the first advertisement delivery strategy is not included in the first advertisement set, a non-experimental advertisement is selected from the non-experimental advertisements in the first advertisement set for delivery.

[0202] Optionally, the advertising experiment system stores setting parameter templates for each experimental quantity in the advertising delivery strategy;

[0203] The first unit 701 can be specifically used for:

[0204] Combining multiple parameter groups of each experimental variable in the experimental element with setting parameter templates corresponding to the experimental quantity to obtain multiple advertising delivery strategies;

[0205] Create experimental ads based on each ad delivery strategy.

[0206] Optionally, the experimental elements also include at least one of an experimental budget, an experimental cycle, and an advertising effect determination index.

[0207] Optionally, the experimental elements also include the application traffic of each parameter group in each traffic layer.

[0208] The advertising experiment device provided in this embodiment belongs to the same application concept as the advertising experiment method provided in the above embodiments of this application, and can execute the advertising experiment method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the advertising experiment method. For technical details not fully described in this embodiment, please refer to the specific processing content of the advertising experiment method provided in the above embodiments of this application, and will not be repeated here.

[0209] The functions implemented by the above first unit 701, the second unit 702 and the third unit 703 can be implemented by the same or different processors respectively, which is not limited in the embodiment of the present application.

[0210] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0211] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0212] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0213] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0214] Exemplary Electronic Devices

[0215] The present application also provides an electronic device, see Figure 8 As shown, the device includes:

[0216] Memory 200 and processor 210;

[0217] The memory 200 is connected to the processor 210 and is used to store programs;

[0218] The processor 210 is used to implement the advertising experiment method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0219] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0220] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0221] A bus may include a pathway that transfers information between components of a computer system.

[0222] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0223] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0224] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0225] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0226] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0227] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0228] The processor 210 executes the program stored in the memory 200 and calls other devices, and can be used to implement each step of any advertising experiment method provided in the above embodiments of the present application.

[0229] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the advertising experiment method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the introduction to the embodiment of the above advertising experiment method.

[0230] Exemplary computer program products and storage media

[0231] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the advertising experimental method according to various embodiments of the present application described in any of the above-mentioned embodiments of this specification.

[0232] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0233] In addition, the embodiment of the present application may also be a storage medium on which a computer program is stored. The computer program is executed by a processor to execute the steps of the advertising experiment method according to various embodiments of the present application described in any of the above embodiments of this specification, and specifically the following steps may be implemented:

[0234] S201. Create multiple experimental advertisements according to the experimental elements of the advertising delivery experiment, wherein the experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable, and the advertising delivery strategies corresponding to the various experimental advertisements are different, wherein one advertising delivery strategy includes a parameter group corresponding to each experimental variable.

[0235] S202. According to a preset traffic allocation strategy, determine a first advertisement delivery strategy corresponding to a first traffic request in the advertisement delivery strategies corresponding to each experimental advertisement, where the first traffic request is used to trigger advertisement delivery.

[0236] S203: Place a first experimental advertisement corresponding to the first advertisement placement strategy, and collect experimental advertisement placement data corresponding to the first traffic request, wherein the experimental advertisement placement data represents an advertisement effect.

[0237] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0238] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0239] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0240] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0241] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0242] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0243] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0244] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0245] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0246] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0247] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An advertising experimental method, characterized in that: Applied to an advertising experiment system, the method includes: Creating multiple experimental ads according to the experimental elements of the advertising delivery experiment, wherein the experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable, and the advertising delivery strategies corresponding to the experimental ads are different, wherein one advertising delivery strategy includes a parameter group corresponding to each experimental variable; According to a preset traffic allocation strategy, determining a first advertisement delivery strategy corresponding to a first traffic request in the advertisement delivery strategies corresponding to each experimental advertisement, wherein the first traffic request is used to trigger advertisement delivery; A first experimental advertisement corresponding to the first advertisement delivery strategy is delivered, and experimental advertisement delivery data corresponding to the first traffic request is collected, wherein the experimental advertisement delivery data represents an advertisement effect.

2. The method according to claim 1, characterized in that The method further comprises: Data analysis is performed on the experimental advertising delivery data corresponding to each traffic request to obtain analysis results, wherein the analysis results include advertising effects corresponding to different parameter groups of the same experimental variable and / or differences in advertising effects of different parameter groups of the same experimental variable.

3. The method according to claim 1, characterized in that The flow distribution strategy includes a layered flow orthogonal strategy, wherein different experimental variables are located in different flow layers, and multiple parameter groups of the same experimental variable are located in the same flow layer; According to the preset traffic allocation strategy, determining the first advertisement delivery strategy corresponding to the first traffic request in the advertisement delivery strategies corresponding to the experimental advertisements includes: According to a preset traffic distribution strategy, determine a parameter group in each traffic layer that matches the first traffic request; The parameter groups matching the first traffic request in each traffic layer are combined to obtain a first advertisement delivery strategy corresponding to the first traffic request.

4. The method according to claim 1, characterized in that: The method further includes: storing a mapping relationship between an advertisement delivery strategy corresponding to a historical traffic request and identification information of the historical traffic request in a mapping relationship library; According to the preset traffic allocation strategy, determining the first advertisement delivery strategy corresponding to the first traffic request in the advertisement delivery strategies corresponding to the experimental advertisements includes: The mapping relationship library is searched using the identification information of the first traffic request, and when it is determined that the mapping relationship library stores a target advertising delivery strategy that has a mapping relationship with the identification information of the first traffic request, the target advertising delivery strategy is determined as the first advertising delivery strategy.

5. The method according to claim 1, characterized in that: The method further comprises: Determine, according to the attribute information of the first traffic request, a first advertisement set in the current advertisement set of the advertisement experiment system, wherein the first traffic request satisfies the advertisement delivery strategy of each advertisement in the first advertisement set, and the current advertisement set includes experimental advertisements and / or non-experimental advertisements; When it is determined that the first advertisement set does not include the first experimental advertisement corresponding to the first advertisement delivery strategy, a non-experimental advertisement is selected from the non-experimental advertisements in the first advertisement set for delivery.

6. The method according to claim 1, characterized in that The advertising experiment system stores setting parameter templates for each experimental quantity in the advertising delivery strategy; Create multiple experimental ads based on the experimental elements of the ad delivery experiment, including: Combining multiple parameter groups of each experimental variable in the experimental element with setting parameter templates corresponding to the experimental quantity of the advertising delivery experiment to obtain multiple advertising delivery strategies; Create experimental ads based on each ad delivery strategy.

7. The method according to claim 1, characterized in that The experimental elements also include at least one of an experimental budget, an experimental cycle, and an advertising effect determination index.

8. The method according to claim 3, characterized in that The experimental elements also include the application traffic of each parameter group in each traffic layer.

9. An advertising experimental device, characterized in that: include: The first unit is used to create multiple experimental advertisements according to the experimental elements of the advertisement delivery experiment, wherein the experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable, and the advertisement delivery strategies corresponding to the experimental advertisements are different, wherein one advertisement delivery strategy includes a parameter group corresponding to each experimental variable; The second unit is used to determine, according to a preset traffic allocation strategy, a first advertisement delivery strategy corresponding to the first traffic request in the advertisement delivery strategies corresponding to each experimental advertisement, wherein the first traffic request is used to trigger the delivery of the advertisement; The third unit is used to deliver the first experimental advertisement corresponding to the first advertisement delivery strategy, and collect experimental advertisement delivery data corresponding to the first traffic request, wherein the experimental advertisement delivery data represents an advertisement effect.

10. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the advertising experiment method according to any one of claims 1 to 8 by running the program in the memory.

11. A computer program product, characterized in that The invention comprises computer program instructions, which, when executed by a processor, enable the processor to implement the advertising experiment method according to any one of claims 1 to 8.

12. An advertising experiment system, characterized in that: include: Server and Client, where The client is used to obtain user input information and send the user input information to the server, wherein the user input information includes experimental elements of an advertising delivery experiment, wherein the experimental elements include at least one experimental variable and multiple parameter groups for each experimental variable, and the experimental variable corresponds to a sub-strategy in the advertising delivery strategy; The server is used to implement the advertising experiment method as described in any one of claims 1-8.

Images