Server, and policy online test method and system

An online test and strategy technology, applied in transmission systems, digital transmission systems, electrical components, etc., can solve the problems of communication flow distribution difficulties, reduce experimental reliability, affect test accuracy, etc., to avoid communication flow distribution difficulties, The overall order and the effect of ensuring accuracy

Active Publication Date: 2016-05-25
ALIBABA (CHINA) CO LTD
View PDF8 Cites 22 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] In view of this, the present invention provides a method and system for online testing of strategies, which solves the difficulties of communication traffic distribution between each strategy team, strong team dependence, and Testing multiple new strategies requires multiple experiments, resulting in a fine distribution of traffic ratios, which reduces the credibility of the experiment and ultimately affects the accuracy of the test.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Server, and policy online test method and system
  • Server, and policy online test method and system
  • Server, and policy online test method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0055] Such as image 3 As shown, it is a schematic flowchart of an embodiment of the method for online testing of strategies in the present invention. In this embodiment, an independent strategy layer is set for each strategy that needs to be tested for testing, and the distribution of test traffic between each strategy layer is independent of each other according to their preset ratios, so that the traffic distribution of each strategy layer has no dependence, avoiding It solves the problem of difficult communication traffic distribution between various strategy teams, and reduces the communication cost between strategy teams. The method for strategy online testing in this embodiment includes the following steps:

[0056] Step 101, receiving a strategy online test request.

[0057] Every new strategy developed offline by technicians needs to be compared with the original strategy before a more favorable strategy can be determined and launched. Because it is difficult to s...

Embodiment 2

[0066] Such as Figure 5 Shown is an optional embodiment of the method for online testing of the strategy described in the embodiment of the present invention. Compared with the flow in the embodiment shown in Embodiment 1, this embodiment optionally describes the process of establishing and adding a strategy testing layer, calculating and distributing traffic of each layer during the online strategy testing process. The method described in this embodiment includes the following steps:

[0067] Step 201, receiving a strategy online test request.

[0068] Step 202: Establish the testing layers of each strategy according to the preset order, and allocate the traffic ratio of the testing buckets in each strategy testing layer.

[0069] Step 203 , when there is a new strategy to be tested, add a new strategy test layer on the basis of the existing strategy layer, and increase the traffic ratio of the test buckets in the strategy test layer according to the preset settings.

[0...

Embodiment 3

[0083] Such as Figure 8A and 8B As shown, it shows an optional embodiment of a strategy online testing method in the invention. In this embodiment, cookies are used as traffic identifiers. In the self-service advertisement delivery system specially built for small and medium advertisers, the algorithm team is responsible for Algorithm design and optimization in the system. Now the algorithm team has two experiments to test the effect of the strategy online: the test of strategy 1 (IdeaSelect) is an experiment of the material selection strategy. An advertisement set by the advertiser often contains 1 to 3 materials, and each delivery can only be from 1 to 3 One of these multiple materials is selected for delivery. Before that, it was a random selection strategy, and each material had an equal chance of being selected. Now I want to add a new strategy 2 (Bid): material selection based on the estimated click-through rate of the material according to the algorithm, and preferen...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a server, and a policy online test method and system. The method comprises the steps of receiving a policy online test request; establishing a test layer for each policy according to a preset sequence, and allocating traffic proportion of a test barrel in each policy test layer; allowing tested traffic to pass through all policy test layers sequentially, and allocating the tested traffic to a corresponding test barrel according to the traffic proportion of the test barrel in each policy test layer so as to perform online test; and collecting test results of test barrels in all policy test layers and comparing the test results, and acquiring an optimal policy of each policy test layer. According to the policy online test method, the problems of the difficulty of communication traffic allocation among all policy teams and strong team dependency are solved, and cost of a plurality of policy tests is lowered.

Description

technical field [0001] The invention relates to the technical field of strategy testing, and relates to a server, a method and a system for online testing of strategies. Background technique [0002] In technical fields such as the Internet, each new strategy is obtained by technicians conducting offline experiments, and whether the newly obtained strategy has an advantage compared with the existing strategy cannot be obtained from offline experiments, because offline experiments It is impossible to simulate the ever-changing real online environment after going online, and it is also impossible to predict who is better between the new strategy and the original strategy. In order to compare the quality of the new strategy with the original strategy and decide whether to replace the original strategy, the online AB test (ABTest) is generally used to test and compare the quality of the new strategy with the original strategy by using a certain amount of traffic. Determining th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/26H04L12/911
CPCH04L43/50H04L47/782
Inventor 雷龙艳章岑杨田周盛潘柏宇王冀
Owner ALIBABA (CHINA) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap