Control experiment method and system, storage medium and equipment
By acquiring and setting custom tags and value ranges, the scalability problem of the AB experimental platform is solved, and flexible configuration and reliable results of the control experiment are realized.
Patent Information
- Application Number
- CN202510594889.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2025-08-22
AI Technical Summary
The experimental audience tags of the existing AB experimental platform rely on upstream systems or database hard codes and cannot be expanded on demand, resulting in a long response cycle of the technical team and traffic overlapping is prone to occur during the experiment.
By obtaining fixed labels for upstream business systems and databases, setting custom labels and value fields, and building association relationships, the scalability of the control experiment is achieved.
It improves the scalability of the control experiment, reduces maintenance costs, ensures accurate hits of traffic and reliability of experimental results, and avoids repeated traffic allocation.
Smart Images
Figure CN120523728A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of business processing technology, and in particular to a control experiment method, system, storage medium and device. Background Art
[0002] In recent years, audience tags in AB experimentation platforms have relied on hard-coded data in upstream systems or databases, making them difficult to scale on demand. Furthermore, adding new tags requires modifying code or database structures before the experiment begins, resulting in a long response time for the technical team (typically 1-3 days). Furthermore, during the experiment itself, tiered experiments require manual assignment of tag values, which can easily lead to traffic overlap (e.g., users hitting two experimental groups simultaneously).
[0003] Therefore, how to improve the scalability of experiments is a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention
[0004] The purpose of this application is to provide a control experiment method, system, computer-readable storage medium and electronic device, which can improve the scalability of control experiments.
[0005] In order to solve the above technical problems, this application provides a control experimental method, and the specific technical solution is as follows:
[0006] When configuring a control experiment, obtain a first fixed tag from the upstream business system and a second fixed tag from the database;
[0007] Setting a custom tag and a value range of the custom tag with reference to the first fixed tag and / or the second fixed tag;
[0008] The custom labels are applied to each layer of the control experiment, and the control experiment is performed.
[0009] Optionally, before setting a custom tag and a value range of the custom tag by referring to the first fixed tag and / or the second fixed tag, the method further includes:
[0010] Configure the tag name and tag type of the custom tag according to the business information of the control experiment;
[0011] Set the value domain type and value domain range of the custom tag.
[0012] Optionally, setting a custom tag and a value range of the custom tag by referring to the first fixed tag and / or the second fixed tag includes:
[0013] determining a first tag rule for the first fixed tag and / or the second fixed tag;
[0014] If the first labeling rule does not meet the control experiment requirements, define a second labeling rule for the custom label with reference to the first labeling rule;
[0015] A custom tag and a value range of the custom tag are set based on the second tag rule.
[0016] Optionally, after setting the custom tag and the value range of the custom tag based on the second tag rule, the method further includes:
[0017] An association relationship is established between the custom tag and the second fixed tag; the association relationship is used to indicate a logical rule and data association between the custom tag and the second fixed tag.
[0018] Optionally, after applying the custom label to each layer of the control experiment, the method further includes:
[0019] Call the rule engine to check whether the current layer has a label and value range combination that is already used by other layers;
[0020] If so, generate error configuration information of the custom tag.
[0021] Optionally, when performing the control experiment, the following steps may also be included:
[0022] Get the traffic distribution strategy;
[0023] Traffic distribution is performed based on the traffic distribution strategy, and a traffic distribution engine is called to assign traffic matching the custom tag to a corresponding experimental layer.
[0024] Optionally, before obtaining the traffic distribution policy, the following steps are also required:
[0025] The traffic allocation strategy is set according to traffic condition allocation rules and traffic priority allocation principles.
[0026] This application also provides a control experimental system, including:
[0027] A fixed tag acquisition module, used to acquire a first fixed tag from an upstream business system and a second fixed tag from a database when configuring a control experiment;
[0028] a label customization module, configured to set a custom label and a value range of the custom label by referring to the first fixed label and / or the second fixed label;
[0029] The experiment configuration module is used to apply the custom label to each layer of the control experiment and execute the control experiment.
[0030] Optionally, also include:
[0031] The custom tag setting module is used to configure the tag name and tag type of the custom tag according to the business information of the control experiment; and set the value domain type and value domain range of the custom tag.
[0032] Optional label customization modules include:
[0033] a first rule determining unit, configured to determine a first tag rule for the first fixed tag and / or the second fixed tag;
[0034] a second rule determining unit, configured to define a second labeling rule for the custom label with reference to the first labeling rule if the first labeling rule does not meet the control experiment requirement;
[0035] The second rule application unit is configured to set a custom tag and a value range of the custom tag based on the second tag rule.
[0036] Optionally, also include:
[0037] The tag association module is used to establish an association relationship between the custom tag and the second fixed tag; the association relationship is used to indicate a logical rule and data association between the custom tag and the second fixed tag.
[0038] Optionally, also include:
[0039] The mutual exclusion check module is used to call the rule engine to detect whether the current layer has a label and value range combination that is already used by other layers; if so, generate error configuration information of the custom label.
[0040] Optionally, when performing the control experiment, the following steps may also be included:
[0041] The traffic distribution module is used to obtain a traffic distribution strategy; perform traffic distribution based on the traffic distribution strategy, and call a traffic distribution engine to assign traffic matching the custom tag to a corresponding experimental layer.
[0042] Optionally, also include:
[0043] The strategy setting module is used to set the traffic distribution strategy according to the traffic condition allocation rule and the traffic priority allocation principle before obtaining the traffic distribution strategy.
[0044] The present application also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-described method when executed by a processor.
[0045] The present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned method when calling the computer program in the memory.
[0046] The present application provides a control experiment method, including: when configuring a control experiment, obtaining a first fixed tag from an upstream business system and a second fixed tag from a database; setting a custom tag and a value range of the custom tag with reference to the first fixed tag and / or the second fixed tag; applying the custom tag to each layer of the control experiment, and executing the control experiment.
[0047] When performing a control experiment, this application first obtains a fixed tag, and sets a custom tag and a corresponding value range with reference to the fixed tag, so that when conducting a control experiment, different custom tags and tag values can be selected to control different traffic hits to achieve traffic distribution and improve the scalability of the business. At the same time, the tag system and business process of the control experiment are structured in the same way, and the iterative process of the business system does not require the reconstruction of the tag logic of the custom tag, which reduces the maintenance cost of the control experiment. On this basis, it is possible to achieve accurate hits on traffic through custom tags, avoid repeated traffic distribution, and improve the reliability of the experimental results of the control experiment.
[0048] The present application also provides a control experiment system, a computer-readable storage medium and an electronic device, which have the above-mentioned beneficial effects and are not described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.
[0050] Figure 1 A flow chart of a control experimental method provided in the examples of the present application;
[0051] Figure 2 Schematic diagram of the existing AB experiment label architecture provided in the embodiment of this application;
[0052] Figure 3 A schematic diagram of custom tag application and traffic hits provided in an embodiment of the present application;
[0053] Figure 4 A schematic diagram of the structure of a control experiment system provided in an embodiment of the present application;
[0054] Figure 5This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0055] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0056] See also Figure 1 , Figure 1 A flow chart of a control experimental method provided in an embodiment of the present application, the method comprising:
[0057] S101: When configuring a control experiment, obtain a first fixed tag from an upstream business system and a second fixed tag from a database;
[0058] S102: Setting a custom tag and a value range of the custom tag with reference to the first fixed tag and / or the second fixed tag;
[0059] S103: Apply the custom label to each layer of the control experiment and execute the control experiment.
[0060] For a better understanding of this application, please see Figure 2 , Figure 2 This is a schematic diagram of the existing AB experiment tag architecture provided in the embodiment of this application. Figure 2 As can be seen, experimental audience tags rely on hard-coded data in upstream systems or databases and cannot be dynamically expanded on demand. Furthermore, business owners cannot customize experimental dimensions, such as adding temporary activity tags, and must rely on technical team intervention and adjustments, making experimental operations extremely inconvenient.
[0061] To solve the above problem, when configuring a control experiment, this application obtains the first fixed tag of the upstream business system and the second fixed tag in the database, and then sets the custom tag and the value range of the custom tag with reference to the first fixed tag and / or the second fixed tag.
[0062] In order to improve the setting efficiency of the custom tag, the tag name and tag type of the custom tag can be configured according to the business information of the control experiment, and then the value domain type and value domain range of the custom tag can be set.
[0063] In actual application, the setting of custom tags can be targeted based on the business needs of the control experiment, so as to selectively set the experimental variable list of the control experiment (such as user grouping, time range or behavioral conditions).
[0064] Custom tag names typically need to reflect their business meaning. They can be categorized into categorical, numeric, and Boolean tags. After setting a custom tag, define the legal value range based on the tag type to avoid data contamination.
[0065] After the custom tags are set, they are applied to each layer of the control experiment, and the control experiment is executed. During the application of each layer of the control experiment, you can manually enter the custom tag rules (including tag name, tag type, and value range) through the management interface, or you can configure the custom tags in code, such as through Python scripts.
[0066] In a feasible implementation, when setting a custom tag, it can be performed as follows:
[0067] Step 1: Determine a first tag rule for the first fixed tag and / or the second fixed tag;
[0068] Step 2: If the first labeling rule does not meet the control experiment requirements, define a second labeling rule for the custom label with reference to the first labeling rule;
[0069] Step 3: Set a custom tag and a value range of the custom tag based on the second tag rule.
[0070] The first labeling rule refers to the labeling rules for fixed labels such as the first and second fixed labels. If the first labeling rule fails to meet the requirements of the control experiment, a second labeling rule for a custom label is defined, and the custom label and its value range are set based on the second labeling rule. It is important to emphasize that the second labeling rule can be set to meet the requirements of the control experiment.
[0071] For example, we can first evaluate the drawbacks of the first labeling rule. By comparing experimental requirements, we can identify areas where existing fixed labels cannot meet the requirements. For example, in a comparative experiment on teaching effectiveness, in addition to recording student names and test scores, we also need to mark the experimental teaching class (classes using different teaching methods) in which the students are enrolled and record their satisfaction with the teaching methods. However, the existing fixed labeling system does not have corresponding labels.
[0072] At this point, the second tag rule for the custom tag is defined with reference to the first tag rule. The advantages of the first tag rule include concise and intuitive tag name definitions, tag type determination based on data properties, and a well-defined value range. For example, the "Student Name" tag has a name that directly indicates its purpose, a character-based tag accurately records names in text form, and a value range consisting of the set of names of students who actually participated in the experiment, which aligns with business practices.
[0073] Finally, based on the second label rule, a custom label and its value range are set. According to the unmet needs of the control experiment and the ideas learned from it, the rules of the custom label are defined. For the custom label that marks the experimental teaching class, the label name is set as "Experimental Teaching Class", the label type is character type, and the value range is the name of each teaching class participating in the experiment, such as "Class A (using teaching method one)" and "Class B (using teaching method two)". For the custom label that records students' satisfaction with the teaching method, the label name is "Teaching Method Satisfaction", the label type is character type, and the value range is "Very Satisfied", "Satisfied", "Average", "Unsatisfied", "Very Unsatisfied" and other options.
[0074] In one feasible implementation, after setting a custom tag and its value range based on the second tag rule, an association relationship can also be established between the custom tag and the second fixed tag. This association relationship indicates the logical rules and data association between the custom tag and the second fixed tag. This facilitates direct definition of the connection between the custom tag and the second fixed tag at the business logic and data levels, enabling more organized data integration and analysis that aligns with business realities.
[0075] Determine the custom tags and second fixed tags to associate. For example, in a controlled experiment evaluating the impact of different training courses on employee performance, you might associate the custom tag "Training Course Category" with the second fixed tag "Employee Performance Score."
[0076] Define logical rules to establish a logical association between custom tags and second fixed tags. This means defining a logical relationship between the two based on actual business needs. For example, a logical rule might be "employee performance scores for different training course categories, used to analyze the effectiveness of training courses on performance improvement."
[0077] In data storage and management systems (such as databases and data warehouses), you can link the custom tag with the fields corresponding to the second fixed tag by writing SQL statements (in the database) or using the association function of data processing tools, based on primary and foreign key relationships (if they exist) or other linkable fields (such as employee ID). For example, in a database, you can use the common field employee ID to join and query the table recording "Training Course Categories" with the table recording "Employee Performance Scores" to achieve a data-level association.
[0078] By linking the two in a database or data report, it's easier to combine the specific business information recorded by custom tags with the basic business data carried by fixed tags. This facilitates subsequent comprehensive analysis and significantly improves the effectiveness of controlled experiments. For example, in an e-commerce controlled experiment, linking the custom tag "Satisfaction with a new promotion method" with the second fixed tag "Purchase amount" allows analysis of the relationship between satisfaction and purchase amount, uncovering the actual impact of promotions on sales.
[0079] At the same time, clarifying the logical relationship between custom tags in business processes and the business elements represented by fixed tags at the business level will help to more accurately describe business scenarios, thereby accurately setting and applying custom tags and avoiding data confusion and misunderstanding.
[0080] When performing a control experiment, the embodiment of the present application first obtains a fixed tag, and sets a custom tag and a corresponding value range with reference to the fixed tag, so that when conducting a control experiment, different custom tags and tag values can be selected to control different traffic hits to achieve traffic distribution and improve the scalability of the business. At the same time, the tag system of the control experiment is structured in the same way as the business process, and the iterative process of the business system does not require reconstruction of the tag logic of the custom tag, which reduces the maintenance cost of the control experiment. On this basis, accurate hits of traffic can be achieved through custom tags, repeated traffic distribution can be avoided, and the reliability of the experimental results of the control experiment can be improved.
[0081] Based on the above examples, existing experimental platforms rely on static tags, which result in tag definitions being fixed in code or databases. Modifications require manual intervention and are prone to errors. Furthermore, user attributes are obtained from upstream business systems as fixed tags, which is limited by the data source structure, resulting in insufficient tag flexibility. Furthermore, experimental tags within the same layer are not systematically verified, leading to a high risk of tag conflicts.
[0082] To solve the above-mentioned defects, after applying the custom label to each layer of the control experiment, this application calls the rule engine to detect whether the current layer has a label and value range combination that has been used by other layers, and generates error configuration information of the custom label when there is a label conflict.
[0083] In the hierarchical experiment configuration stage, automatically detect whether there are duplicate label and value range combinations within the same layer. If a conflict is detected (for example, experiment A has already used user_region = East China, and at the same time experiment B also uses user_region = East China), then prohibit experiment B from creating and using this label value, and give an error prompt. However, if the custom label used by experiment B is user_region = South China, although the label is the same, the value range is different, and there is no combination of label and value range already used in other layers.
[0084] Correspondingly, in the process of traffic hit policy, the traffic distribution policy can be obtained first, so as to perform traffic distribution based on the traffic distribution policy, and call the traffic distribution engine to hit the traffic that meets the custom label to the corresponding experimental layer. <0000
[0092] As can be seen, this embodiment enables zero-code tag management and immediate effectiveness. The rule engine ensures the uniqueness of different experimental tag values within the same layer, thereby ensuring the mutual exclusivity of different experimental traffic within the same layer. Flexible configuration of experimental tags achieves the purpose of traffic distribution and improves business scalability. Different businesses can independently define tag systems (such as product_category for e-commerce and device_type for gaming). This is suitable for scenarios where temporary activities require quick tag configuration (such as campaign_id for holiday promotions) and also facilitates refined grouping based on user behavior tags (such as last_purchase_day).
[0093] See also Figure 4 , Figure 4 A schematic diagram of the structure of a control experiment system provided in an embodiment of the present application, the system includes:
[0094] A fixed tag acquisition module, used to acquire a first fixed tag from an upstream business system and a second fixed tag from a database when configuring a control experiment;
[0095] a label customization module, configured to set a custom label and a value range of the custom label by referring to the first fixed label and / or the second fixed label;
[0096] The experiment configuration module is used to apply the custom label to each layer of the control experiment and execute the control experiment.
[0097] Based on the above embodiment, as a preferred embodiment, it also includes:
[0098] The custom tag setting module is used to configure the tag name and tag type of the custom tag according to the business information of the control experiment; and set the value domain type and value domain range of the custom tag.
[0099] Based on the above embodiment, as a preferred embodiment, the label customization module includes:
[0100] a first rule determining unit, configured to determine a first tag rule for the first fixed tag and / or the second fixed tag;
[0101] a second rule determining unit, configured to define a second labeling rule for the custom label with reference to the first labeling rule if the first labeling rule does not meet the control experiment requirement;
[0102] The second rule application unit is configured to set a custom tag and a value range of the custom tag based on the second tag rule.
[0103] Based on the above embodiment, as a preferred embodiment, it also includes:
[0104] The tag association module is used to establish an association relationship between the custom tag and the second fixed tag; the association relationship is used to indicate a logical rule and data association between the custom tag and the second fixed tag.
[0105] Based on the above embodiment, as a preferred embodiment, it also includes:
[0106] The mutual exclusion check module is used to call the rule engine to detect whether the current layer has a label and value range combination that is already used by other layers; if so, generate error configuration information of the custom label.
[0107] Based on the above embodiment, as a preferred embodiment, when performing the control experiment, the method further includes:
[0108] The traffic distribution module is used to obtain a traffic distribution strategy; perform traffic distribution based on the traffic distribution strategy, and call a traffic distribution engine to assign traffic matching the custom tag to a corresponding experimental layer.
[0109] Based on the above embodiment, as a preferred embodiment, it also includes:
[0110] The strategy setting module is used to set the traffic distribution strategy according to the traffic condition allocation rule and the traffic priority allocation principle before obtaining the traffic distribution strategy.
[0111] The present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in the above method embodiment.
[0112] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.
[0113] The computer-readable storage medium provided in this embodiment includes the above-mentioned method, and the effect is the same as above.
[0114] This application also provides an electronic device, see Figure 5 , a structural diagram of an electronic device provided in an embodiment of the present application, such as Figure 5 As shown, a processor 1410 and a memory 1420 may be included.
[0115] The processor 1410 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1410 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 1410 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1410 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1410 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0116] The memory 1420 may include one or more computer-readable storage media, which may be non-transitory. The memory 1420 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 1420 is at least used to store the following computer program 1421, wherein, after the computer program is loaded and executed by the processor 1410, it can implement the relevant steps in the method performed by the electronic device side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 1420 may also include an operating system 1422 and data 1423, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 1422 may include Windows, Linux, Android, etc.
[0117] In some embodiments, the electronic device may further include a display screen 1430 , an input / output interface 1440 , a communication interface 1450 , a sensor 1460 , a power supply 1470 , and a communication bus 1480 .
[0118] certainly, Figure 5 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Figure 5 More or fewer components than shown, or combinations of certain components.
[0119] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems provided in the embodiments, since they correspond to the methods provided in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.
[0120] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core ideas of this application. It should be noted that for those skilled in the art, without departing from the principles of this application, various improvements and modifications can be made to this application, and such improvements and modifications also fall within the scope of protection of this application.
[0121] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.
Claims
1. A control experimental method, characterized in that: include: When configuring a control experiment, obtain a first fixed tag from the upstream business system and a second fixed tag from the database; Setting a custom tag and a value range of the custom tag with reference to the first fixed tag and / or the second fixed tag; The custom labels are applied to each layer of the control experiment, and the control experiment is performed.
2. The control experimental method according to claim 1, characterized in that Before setting a custom tag and a value range of the custom tag by referring to the first fixed tag and / or the second fixed tag, the method further includes: Configure the tag name and tag type of the custom tag according to the business information of the control experiment; Set the value domain type and value domain range of the custom tag.
3. The control experiment method according to claim 2, characterized in that Setting a custom tag and a value range of the custom tag by referring to the first fixed tag and / or the second fixed tag includes: determining a first tag rule for the first fixed tag and / or the second fixed tag; If the first labeling rule does not meet the control experiment requirements, define a second labeling rule for the custom label with reference to the first labeling rule; A custom tag and a value range of the custom tag are set based on the second tag rule.
4. The control experimental method according to claim 3, characterized in that After setting the custom tag and the value range of the custom tag based on the second tag rule, the method further includes: An association relationship is established between the custom tag and the second fixed tag; the association relationship is used to indicate a logical rule and data association between the custom tag and the second fixed tag.
5. The control experiment method according to claim 1, characterized in that After applying the custom labels to each layer of the control experiment, the method further includes: Call the rule engine to check whether the current layer has a label and value range combination that is already used by other layers; If so, generate error configuration information of the custom tag.
6. The control experimental method according to claim 1, characterized in that When performing the control experiment, the following steps are also included: Get the traffic distribution strategy; Traffic distribution is performed based on the traffic distribution strategy, and a traffic distribution engine is called to assign traffic matching the custom tag to a corresponding experimental layer.
7. The control experimental method according to claim 6, characterized in that: Before obtaining the traffic distribution strategy, the following steps are also required: The traffic allocation strategy is set according to traffic condition allocation rules and traffic priority allocation principles.
8. A control experimental system, characterized in that: include: A fixed tag acquisition module, used to acquire a first fixed tag from an upstream business system and a second fixed tag from a database when configuring a control experiment; a label customization module, configured to set a custom label and a value range of the custom label by referring to the first fixed label and / or the second fixed label; The experiment configuration module is used to apply the custom label to each layer of the control experiment and execute the control experiment.
9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which implements the steps of the method according to any one of claims 1 to 7 when executed.