Experimental platform load balancing method, computing equipment and storage media

Abstract

This invention discloses a traffic distribution method, computing device, and storage medium for an experimental platform. The method is executed in the computing device. The experimental platform includes an experimental area and a baseline area, each comprising multiple experimental layers. The method includes: in response to a traffic allocation request, obtaining a list of experimental classes on the experimental platform; obtaining a bucket identifier from the traffic allocation request, and obtaining a corresponding experimental class bucket from the list of experimental classes based on the bucket identifier, wherein the experimental class bucket is suitable for being divided into a first predetermined number of experimental buckets; obtaining user information and performing a hash calculation to obtain a first hash value; performing a modulo operation on the first hash value and the first predetermined number of buckets to obtain a first remainder; and allocating traffic to target experimental buckets in the experimental area or the baseline area based on the first remainder. According to the technical solution of this invention, experiments in the baseline area can serve as a control for experiments in the experimental area, facilitating comparison of experimental results.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and in particular to an experimental platform traffic splitting method, computing device, and storage medium. Background Technology

[0002] For today's high-traffic internet platforms, even a minor page redesign or a small modification to the parameters of a backend content recommendation model can have a significant impact, necessitating the use of experimental platforms to validate these improvements. Currently, common experimental platforms are single-layered, with simple logic, such as hashing userid / deviceid and then taking the modulo to distribute traffic into 100 portions from 0 to 99. Multi-layered experiments, building upon the mutual exclusion relationships of single-layered experiments, add orthogonal relationships between experiments, enriching the application of experimental methods.

[0003] However, as the business scenarios and needs of the experimental platform become increasingly refined and diverse, the existing experimental platform has gradually exposed the following problems: a large number of experiments affect business-related metrics, making it unclear which experiments are truly positive; when expanding or shrinking the scale of experiments, problems may occur in the data for that day, with previously selected control buckets now becoming experimental buckets; there is a lack of complete rules to confirm the effectiveness of experimental buckets, making it impossible to confirm whether the scale can be expanded; the aggregated business metrics are affected by the metrics of the experiments, resulting in a comprehensive result, and the true logical metrics of the default are unclear, lacking a comparison of all experiments.

[0004] Therefore, an experimental platform diversion method is needed to solve the problems existing in the above scheme. Summary of the Invention

[0005] Therefore, the present invention provides an experimental platform diversion method to solve or at least alleviate the above-mentioned problems.

[0006] According to one aspect of the present invention, a method for traffic splitting in an experimental platform is provided, executed in a computing device. The experimental platform includes an experimental area and a baseline area, each of which includes multiple experimental layers, each experimental layer being adapted to perform one or more experiments. The method includes: in response to a traffic allocation request, obtaining a list of experimental classes of the experimental platform; obtaining a bucket identifier from the traffic allocation request, and obtaining a corresponding experimental class bucket from the list of experimental classes based on the bucket identifier, wherein the experimental class bucket is adapted to be divided into a first predetermined number of experimental buckets; obtaining user information and performing a hash calculation to obtain a first hash value; performing a modulo calculation on the first hash value and the first predetermined number of buckets to obtain a first remainder; and allocating the traffic to a target experimental bucket in the experimental area or the baseline area based on the first remainder.

[0007] Optionally, in the experimental platform traffic splitting method according to the present invention, allocating the traffic to target experimental buckets in the experimental area or baseline area according to the first remainder includes: determining whether the first remainder is less than a second predetermined number of buckets; if the first remainder is less than the second predetermined number of buckets, then allocating the traffic to target experimental buckets in the experimental area; if the first remainder is not less than the second predetermined number of buckets, then allocating the traffic to target experimental buckets in the baseline area; wherein, the first predetermined number of buckets is twice the second predetermined number of buckets.

[0008] Optionally, in the experimental platform traffic splitting method according to the present invention, allocating the traffic to the target experimental bucket in the experimental area includes: traversing each experimental layer in the experimental area and obtaining the experimental list of the experimental layer; obtaining the experimental layer information and performing hash calculation to obtain a second hash value; performing modulo calculation on the second predetermined number of buckets according to the second hash value to obtain a second remainder; determining the target experimental bucket of the experimental layer according to the second remainder, and allocating the traffic to the target experimental bucket of the experimental layer.

[0009] Optionally, in the experimental platform traffic splitting method according to the present invention, allocating the traffic to target experimental buckets in the baseline area includes: traversing each experimental layer in the baseline area; obtaining a list of experimental buckets for each experimental layer as target experimental buckets in the baseline area; and allocating the traffic to the target experimental buckets in the baseline area.

[0010] Optionally, in the experimental platform traffic splitting method according to the present invention, the method further includes: determining the click rate of each experimental bucket based on a predetermined experimental period; if the click rate of the experimental bucket is greater than or equal to a target value, then expanding the quantity of the experimental bucket.

[0011] Optionally, in the experimental platform traffic splitting method according to the present invention, the method further includes: if the click rate of the experimental bucket is less than the target value, then the experimental bucket is reduced in size.

[0012] Optionally, in the experimental platform splitting method according to the present invention, the method further includes: obtaining all target experimental buckets in the experimental area or baseline area, generating an experimental bucket set; and requesting experimental services based on the experimental bucket set.

[0013] Optionally, in the experimental platform traffic splitting method according to the present invention, the user information includes user identifier and date.

[0014] Optionally, in the experimental platform traffic splitting method according to the present invention, the experimental layer information includes visitor count, date, and experimental creation time.

[0015] Optionally, in the experimental platform diversion method according to the present invention, the first predetermined number of buckets is 100, and the second predetermined number of buckets is 50.

[0016] According to one aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the experimental platform shunting method as described above.

[0017] According to one aspect of the present invention, a readable storage medium storing program instructions is provided, which, when read and executed by a computing device, causes the computing device to perform the experimental platform shunting method as described above.

[0018] According to the technical solution of the present invention, a method for traffic distribution on an experimental platform is provided, wherein an experimental area and a baseline area are divided, and the experimental area and the baseline area each include corresponding multiple experimental layers. In response to a traffic allocation request, a list of experimental classes of the experimental platform is obtained; a bucket identifier is obtained from the traffic allocation request; and a corresponding experimental class bucket is obtained from the list of experimental classes based on the bucket identifier. User information is obtained and hashed to obtain a first hash value; the first hash value is modulo-calculated with a first predetermined number of buckets to obtain a first remainder. Based on the first remainder, traffic is allocated to target experimental buckets in the experimental area or the baseline area. If the first remainder is less than a second predetermined number of buckets, the traffic is allocated to the target experimental bucket in the experimental area; otherwise, the traffic is allocated to the baseline area. Thus, according to the technical solution of the present invention, all traffic can be evenly distributed between the experimental area and the baseline area, and all traffic can be distributed to multiple experimental buckets, ensuring that 50% of the traffic is used only for experiments in the baseline area. In this way, experiments in the baseline area can serve as a control for experiments in the experimental area, facilitating comparison of experimental results.

[0019] Furthermore, according to the technical solution of the present invention, after the traffic allocation is completed, the click-through rate of each experimental bucket can be determined based on a predetermined experimental period. If the click-through rate of the experimental bucket is greater than or equal to the target value, it can be determined as positive, and the traffic of the experimental bucket is increased until it is fully pushed; otherwise, the traffic is automatically reduced until it is closed. In this way, the present invention can clearly determine the effect of the experimental bucket and determine whether the traffic can be increased.

[0020] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0021] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of this disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout this disclosure, the same reference numerals generally refer to the same parts or elements.

[0022] Figure 1 A schematic diagram of a computing device 100 according to an embodiment of the present invention is shown;

[0023] Figure 2 A flowchart of an experimental platform load splitting method 200 according to an embodiment of the present invention is shown;

[0024] Figure 3 A schematic diagram of the partitioning of an experimental platform according to an embodiment of the present invention is shown;

[0025] Figure 4 A flowchart of step 250 according to an embodiment of the present invention is shown. Detailed Implementation

[0026] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0027] Figure 1 A schematic diagram of a computing device 100 according to an embodiment of the present invention is shown. Figure 1 As shown, in a basic configuration, computing device 100 includes at least one processing unit 102 and system memory 104. According to one aspect, depending on the configuration and type of the computing device, the processing unit 102 may be implemented as a processor. System memory 104 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, system memory 104 includes an operating system 105.

[0028] According to one aspect, operating system 105 is, for example, suitable for controlling the operation of computing device 100. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. Figure 1The basic configuration is illustrated by the components within the dashed lines. According to one aspect, the computing device 100 has additional features or functions. For example, according to one aspect, the computing device 100 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 1 The image is shown by removable storage device 109 and non-removable storage device 110.

[0029] As stated above, according to one aspect, a program module 103 is stored in system memory 104. According to one aspect, program module 103 may include one or more applications. The invention does not limit the type of application; for example, applications may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browser applications, etc. In an embodiment of the invention, program module 103 includes one or more program instructions for executing the experimental platform routing method 200 of the invention, these program instructions being configured to execute the experimental platform routing method 200 of the invention.

[0030] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 1 Each or many of the components shown can be implemented as an example by integrating a System-on-a-Chip (SOC) on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 100. Embodiments of the invention can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of the invention can be implemented within a general-purpose computer or in any other circuit or system.

[0031] According to one aspect, computing device 100 may also have one or more input devices 112, such as a keyboard, mouse, pen, voice input device, touch input device, etc. It may also include output devices 114, such as a display, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. Computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0032] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program module 103). System memory 104, removable storage device 109, and non-removable storage device 110 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computer device 100. According to one aspect, any such computer storage medium can be part of computing device 100. Computer storage media does not include carrier waves or other transmitted data signals.

[0033] According to one aspect, the communication medium is implemented by computer-readable instructions, data structures, program modules 103, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, the communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0034] In an embodiment of the invention, computing device 100 is configured to execute experimental platform offloading method 200 according to the invention. Computing device 100 includes one or more processors and one or more readable storage media storing program instructions that, when configured to be executed by the one or more processors, cause the computing device to execute the experimental platform offloading method 200 of the present invention.

[0035] Figure 2 A flowchart of an experimental platform offloading method 200 according to an embodiment of the present invention is shown. Method 200 is adapted to be executed in a computing device (the aforementioned computing device 100). The computing device 100 includes an experimental platform.

[0036] Figure 3 A schematic diagram of the experimental platform partitioning according to an embodiment of the present invention is shown. Figure 3 As shown, the experimental platform includes an experimental area and a baseline area (base area). According to an embodiment of the present invention, all traffic can be evenly divided into the experimental area and the baseline area. The experimental area and the baseline area can each be divided into multiple experimental layers. Each experimental layer in the experimental area corresponds to one experimental layer in the baseline area. Each experimental layer can perform one or more experiments, and each experiment can be configured with one or more experimental buckets. Each experimental bucket can be allocated a corresponding proportion of traffic.

[0037] It should be noted that all experiments in the experimental zone collectively affect the final experimental result, while the experimental results in the baseline zone are unaffected by ongoing experiments. The experimental zone and the baseline zone are mutually exclusive, as are multiple experiments within the same experimental layer and multiple experimental buckets assigned to the same experiment. Different experimental layers can correspond to different business modules to meet various business needs. Furthermore, this invention distinguishes between experimental logic and default logic by dividing the experimental zone and the base zone, facilitating the comparison of experimental results.

[0038] like Figure 2 As shown, method 200 includes steps 210 to 250.

[0039] In step 210, a user's traffic allocation request is received. In response to the traffic allocation request, a list of experimental classes for the experimental platform is obtained.

[0040] Subsequently, in step 220, the bucket identifier is obtained from the traffic allocation request, and the class experiment bucket corresponding to the bucket identifier is obtained from the class experiment list according to the bucket identifier.

[0041] It should be noted that, in the embodiments of the present invention, experiments can be classified, and different types of experiments can be configured and use different class experiment buckets. In other words, each type of experiment can correspond to one class experiment bucket.

[0042] In one embodiment of the present invention, a class of experimental buckets may include multiple experimental buckets. In other words, a class of experimental buckets may be divided into multiple experimental buckets. For example, a class of experimental buckets may be divided into a first predetermined number of experimental buckets so that the first predetermined number of experimental buckets can be allocated to multiple experiments in each experimental layer in the experimental area and the baseline area.

[0043] It should be noted that the class experiment list can include multiple class experiment buckets, and each class experiment bucket can include multiple user information items of the corresponding type. Furthermore, each class experiment bucket includes a bucket identifier, allowing the corresponding class experiment bucket (target class experiment bucket) to be selected from the class experiment list based on the bucket identifier.

[0044] In one embodiment, the class experiment list is, for example, a UI class experiment list, and correspondingly, the class experiment bucket can be a UI class experiment bucket.

[0045] Subsequently, in step 230, user information is obtained from the traffic allocation request, and the user information is hashed to obtain the first hash value of the user information.

[0046] In one embodiment, user information may include, for example, a user identifier (or device identifier) ​​and a date.

[0047] Next, step 240 is performed for the first hash value of each user's information.

[0048] In step 240, the first remainder (A) is obtained by performing a modulo operation on the first predetermined number of buckets based on the first hash value.

[0049] In one embodiment, the first predetermined number of buckets is, for example, 100.

[0050] Finally, in step 250, the flow rate is allocated to the target experimental bucket in the experimental area or baseline area based on the first remainder.

[0051] According to one embodiment of the present invention, half of the first predetermined number of buckets is determined as the second predetermined number of buckets; in other words, the first predetermined number of buckets is twice the second predetermined number of buckets. When the first predetermined number of buckets is 100, the second predetermined number of buckets is correspondingly 50.

[0052] Figure 4 A flowchart of step 250 according to an embodiment of the present invention is shown. Figure 4 As shown, in step 250, the specific method for allocating the flow rate to the target experimental buckets in the experimental area or baseline area based on the first remainder is as follows:

[0053] First, determine whether the first remainder is less than the second predetermined number of buckets.

[0054] If the first remainder is less than the second predetermined number of buckets, the traffic will be allocated to the target experimental bucket in the experimental area.

[0055] Specifically, iterate through each experimental layer in the experimental area, and for each experimental layer, obtain the experimental list of the experimental layer.

[0056] The experimental layer information is retrieved and hashed to obtain a second hash value. Here, the experimental layer information may include, for example, visitor count (UV), date, and experiment creation time. Visitor count (UV) can be replaced with pageview count (PV).

[0057] Subsequently, the second remainder (B) is obtained by modulo the second hash value with the second predetermined number of buckets.

[0058] Based on the second remainder, the target experimental bucket for the experimental layer is determined, and traffic is allocated to that target experimental bucket. Here, the experimental layer can include multiple experiments, and each experiment can be configured with one or more experimental buckets, specifically allocating traffic to the target experimental bucket of the target experiment within the experimental layer.

[0059] Additionally, if the first remainder is not less than the second predetermined number of buckets, traffic can be allocated to the target experimental bucket in the baseline area. Specifically, this can be executed according to the default logic.

[0060] Specifically, by traversing each experimental layer in the baseline region, a list of main traffic experimental buckets for each experimental layer is obtained. All experimental buckets in the list of experimental buckets for each experimental layer are used as target experimental buckets in the baseline region, and traffic is allocated to each target experimental bucket in the baseline region. In this way, the baseline region metrics for each experimental layer remain consistent and are not affected by ongoing experiments, so as to provide a control object for the experiments of each experimental layer in the experimental region.

[0061] By executing the above method, and allocating traffic to target experimental buckets in the experimental or baseline areas based on the first remainder, it is possible to evenly distribute all traffic between the experimental and baseline areas, and to distribute all traffic to multiple experimental buckets. This ensures that 50% of the traffic is used exclusively for experiments in the baseline area.

[0062] According to one embodiment of the present invention, after the traffic allocation is completed in step 250, the click-through rate (CTR) of each experimental bucket can be determined based on a predetermined experimental period. Here, the click-through rate CTR = (number of clicks / number of impressions) * 100%. The predetermined experimental period may, for example, include N days.

[0063] Next, it is determined whether the click-through rate (CTR) of the experimental bucket is greater than or equal to the target value. If the CTR of the experimental bucket is greater than or equal to the target value, it can be considered a positive result, and the experimental bucket is expanded until it is fully pushed. This ensures that all experiments need to be verified by traffic in the experimental area before being expanded to the full push.

[0064] Additionally, if the click-through rate of the experimental bucket is less than the target value, the experimental bucket will be scaled down until it is closed.

[0065] It should be noted that the present invention does not limit the specific value of the target value, which can be set by those skilled in the art according to actual needs.

[0066] In one embodiment, the target value is, for example, 0.2%.

[0067] Additionally, according to one embodiment of the present invention, after executing step 250, the following steps may also be performed: obtaining all target experimental buckets in the experimental area or baseline area, and generating an experimental bucket set. Then, based on the experimental bucket set, downstream experimental services can be requested for each target experimental bucket in the experimental bucket set. Downstream experimental services include, for example, one or more experimental service modules (stages) selected from: indexing, recall, sorting, forward ranking, feature generation, intervention, and packaging. Next, each experimental service module can be divided into different levels according to different usage scenarios. Each level can be expanded, with experiments at the same level mutually exclusive, and buckets within the same experiment mutually exclusive.

[0068] According to the experimental platform traffic distribution method of the present invention, an experimental area and a baseline area are divided, each including multiple corresponding experimental layers. In response to a traffic allocation request, a list of experimental classes of the experimental platform is obtained; a bucket identifier is obtained from the traffic allocation request; and the corresponding experimental bucket is obtained from the list of experimental classes based on the bucket identifier. User information is obtained and hashed to obtain a first hash value; the first hash value is modulo-calculated with a first predetermined number of buckets to obtain a first remainder. Based on the first remainder, traffic is allocated to target experimental buckets in the experimental area or the baseline area. If the first remainder is less than a second predetermined number of buckets, the traffic is allocated to the target experimental bucket in the experimental area; otherwise, the traffic is allocated to the baseline area. Thus, according to the technical solution of the present invention, all traffic can be evenly distributed between the experimental area and the baseline area, and all traffic can be distributed to multiple experimental buckets, ensuring that 50% of the traffic is used only for experiments in the baseline area. In this way, experiments in the baseline area can serve as a control for experiments in the experimental area, facilitating comparison of experimental results.

[0069] Furthermore, according to the technical solution of the present invention, after the traffic allocation is completed, the click-through rate of each experimental bucket can be determined based on a predetermined experimental period. If the click-through rate of the experimental bucket is greater than or equal to the target value, it can be determined as positive, and the traffic of the experimental bucket is increased until it is fully pushed; otherwise, the traffic is automatically reduced until it is closed. In this way, the present invention can clearly determine the effect of the experimental bucket and determine whether the traffic can be increased.

[0070] A9. The method as described in A3, wherein the experimental layer information includes visitor count, date, and experiment creation time. A10. The method as described in any one of A2-A8, wherein the first predetermined number of buckets is 100, and the second predetermined number of buckets is 50.

[0071] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0072] When the program code is executed on a programmable computer, the mobile terminal generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the experimental platform shunt method of the present invention according to instructions in the program code stored in the memory.

[0073] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program module 103, or other data. Communication media generally embodies computer-readable instructions, data structures, program module 103, or other data in the form of modulated data signals such as a carrier wave or other transmission mechanism, and includes any information transmission medium. Any combination of the above is also included within the scope of readable media.

[0074] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0075] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0076] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more aspects of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0077] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0078] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0079] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0080] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the methods or method elements forms means for implementing the methods or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by elements for the purposes of carrying out the invention.

[0081] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

[0082] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the invention is illustrative rather than restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. A method for offloading experiments onto a computing device, wherein the experimental platform includes an experimental area and a baseline area, the experimental area and the baseline area each including multiple experimental layers, each experimental layer being adapted to perform one or more experiments, and the experiments in the baseline area being adapted to serve as a control for the experiments in the experimental area, the method comprising: In response to a traffic allocation request, the experimental platform's class experiment list is obtained. The class experiment list includes multiple class experiment buckets, and each class experiment bucket includes a bucket identifier. Obtain the bucket identifier from the traffic allocation request, and obtain the corresponding class experiment bucket from the class experiment list according to the bucket identifier. Each type of experiment corresponds to one class experiment bucket, and the class experiment bucket is suitable for being divided into a first predetermined number of experiment buckets. Obtain user information and perform hash calculation to obtain a first hash value. The user information includes user identifier and date. The first remainder is obtained by performing a modulo operation on the first predetermined number of buckets based on the first hash value. Determine whether the first remainder is less than the second predetermined number of buckets, wherein the first predetermined number of buckets is twice the second predetermined number of buckets; If the first remainder is less than the second predetermined number of buckets, the traffic is allocated to the target experimental bucket in the experimental area, including: traversing each experimental layer in the experimental area and obtaining the experimental list of the experimental layer; obtaining the experimental layer information and performing hash calculation to obtain a second hash value, wherein the experimental layer information includes visitor count, date, and experimental creation time; performing modulo calculation on the second predetermined number of buckets based on the second hash value to obtain a second remainder; determining the target experimental bucket of the experimental layer based on the second remainder, and allocating the traffic to the target experimental bucket of the experimental layer. If the first remainder is not less than the second predetermined number of buckets, the traffic is allocated to the target experimental buckets in the baseline area, including: traversing each experimental layer in the baseline area, obtaining the experimental bucket list of each experimental layer as the target experimental buckets in the baseline area, and allocating the traffic to the target experimental buckets in the baseline area. Based on the predetermined experimental period, the click rate of each experimental bucket is determined; If the click-through rate of the experimental bucket is greater than or equal to the target value, then the experimental bucket will be expanded. If the click-through rate of the experimental bucket is less than the target value, the experimental bucket will be scaled down.

2. The method as described in claim 1, wherein, The method further includes: Obtain all target experimental buckets in the experimental area or baseline area, and generate an experimental bucket set; Request experimental services based on the set of experimental buckets.

3. The method as described in claim 1 or 2, wherein, The first predetermined number of buckets is 100, and the second predetermined number of buckets is 50.

4. A computing device, comprising: At least one processor; as well as A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the method as described in any one of claims 1-3.

5. A readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the method as described in any one of claims 1-3.

Images