Experiment shunting method and device, electronic equipment and storage medium

Abstract

The invention provides an experiment shunting method and device, electronic equipment and a storage medium, and relates to the technical field of data analysis, in particular to the technical field of experiment shunting and application development. The method comprises the steps that in response to the fact that first request flow meets the shunting condition of an experiment, information of the first request flow and configuration information of the experiment are obtained, and the experiment comprises a plurality of candidate experiment versions; extracting an identification of the experiment and scene information of the experiment from the configuration information of the experiment; based on the information of the first request flow, the identification of the experiment and the scene information of the experiment, determining a target experiment version from the plurality of candidate experiment versions; and distributing the first request traffic to a traffic group corresponding to the target experiment version, and storing a shunting result of the first request traffic to a local storage space.

Description

Technical Field

[0001] The present disclosure relates to the field of data analysis technology, and particularly to the fields of experimental shunting and application development technology. In particular, it relates to an experimental shunting method, device, electronic device, storage medium, and computer program product. Background Art

[0002] AB testing (A / B Testing), also known as split testing or controlled experiment, is a statistical method used to compare the differences between multiple versions (such as version A and version B) to determine which version performs better on specific metrics. Experimental shunting is one of the key technologies in AB testing and multivariate testing, and is used to randomly or regularly allocate request traffic to the experimental group or the control group.

[0003] However, in the related art, there are problems of low uniformity and orthogonality in experimental shunting, resulting in low accuracy of experimental results. Summary of the Invention

[0004] The present disclosure provides an experimental shunting method, device, electronic device, storage medium, and computer program product.

[0005] According to a first aspect of the present disclosure, an experimental shunting method is provided, including: in response to a first request traffic satisfying the shunting condition of an experiment, obtaining information of the first request traffic and configuration information of the experiment, where the experiment includes multiple candidate experiment versions; extracting an identifier of the experiment and scenario information of the experiment from the configuration information of the experiment; determining a target experiment version from the multiple candidate experiment versions based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment; allocating the first request traffic to a traffic group corresponding to the target experiment version, and storing the shunting result of the first request traffic in a local storage space.

[0006] According to a second aspect of the present disclosure, another experimental shunting method is provided, including: adding an identifier of the experiment and the scenario information of the experiment to the configuration information of the experiment, where the experiment includes multiple candidate experiment versions, and the identifier of the experiment and the scenario information of the experiment are used to determine a target experiment version to be tested for a first request traffic from the multiple candidate experiment versions; sending the configuration information of the experiment to a service end according to the transmission mode of the pass-through parameter.

[0007] According to a third aspect of the present disclosure, an experimental traffic shunting device is provided, including: an acquisition module, configured to acquire information of the first request traffic and configuration information of the experiment in response to the first request traffic meeting the traffic shunting condition of the experiment, where the experiment includes a plurality of candidate experiment versions; an extraction module, configured to extract an identifier of the experiment and scenario information of the experiment from the configuration information of the experiment; a determination module, configured to determine a target experiment version from the plurality of candidate experiment versions based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment; and a storage module, configured to allocate the first request traffic to a traffic group corresponding to the target experiment version and store the traffic shunting result of the first request traffic in a local storage space.

[0008] According to a fourth aspect of the present disclosure, another experimental traffic shunting device is provided, including: an addition module, configured to add an identifier of the experiment and the scenario information of the experiment to the configuration information of the experiment, where the experiment includes a plurality of candidate experiment versions, and the identifier of the experiment and the scenario information of the experiment are used to determine a target experiment version to be tested by the first request traffic from the plurality of candidate experiment versions; and a sending module, configured to send the configuration information of the experiment to a service end according to a transmission mode of pass-through parameters.

[0009] According to a fifth aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; where the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the experimental traffic shunting method proposed in the first aspect above, and / or execute the experimental traffic shunting method proposed in the second aspect above.

[0010] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause the computer to execute the experimental traffic shunting method proposed in the first aspect above, and / or execute the experimental traffic shunting method proposed in the second aspect above.

[0011] According to a seventh aspect of the present disclosure, a computer program product is provided, including a computer program, where the computer program, when executed by a processor, implements the experimental traffic shunting method proposed in the first aspect above, and / or implements the experimental traffic shunting method proposed in the second aspect above.

[0012] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0013] The accompanying drawings are used to better understand the present solution and do not constitute a limitation to the present disclosure. Among them:

[0014] Figure 1 It is a schematic flowchart of an experimental shunting method according to an embodiment of the present disclosure;

[0015] Figure 2 It is a schematic flowchart of an experimental shunting method according to another embodiment of the present disclosure;

[0016] Figure 3 It is a schematic flowchart of an experimental shunting method according to another embodiment of the present disclosure;

[0017] Figure 4 It is a schematic flowchart of an experimental shunting method according to another embodiment of the present disclosure;

[0018] Figure 5 It is a schematic flowchart of an experimental shunting method according to another embodiment of the present disclosure;

[0019] Figure 6 It is a schematic flowchart of an experimental shunting method according to another embodiment of the present disclosure;

[0020] Figure 7 It is a schematic flowchart of an experimental shunting method according to another embodiment of the present disclosure;

[0021] Figure 8 It is a schematic flowchart of an experimental shunting method according to another embodiment of the present disclosure;

[0022] Figure 9 It is a schematic diagram of a service end and an experimental platform according to an embodiment of the present disclosure;

[0023] Figure 10 It is a schematic structural diagram of an experimental shunting device according to an embodiment of the present disclosure;

[0024] Figure 11 It is a schematic structural diagram of an experimental shunting device according to another embodiment of the present disclosure;

[0025] Figure 12 It is a schematic block diagram of an electronic device according to an embodiment of the present disclosure. Specific embodiments

[0026] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] Data analysis refers to the analysis of a large amount of collected data using appropriate statistical analysis methods, summarizing, understanding, and digesting them to maximize the development of the data's functions and play the role of the data. Data analysis is a process of detailed research and summary of data to extract useful information and form conclusions.

[0028] Experimental traffic splitting is one of the key technologies in A / B testing and multivariate testing, used to randomly or according to rules distribute request traffic to the experimental group or the control group. This process ensures that each request traffic is fairly distributed to the experimental group or the control group, so that the effects of different versions can be accurately evaluated. A / B testing, also known as split testing or controlled experiment, is a statistical method used to compare the differences between multiple versions (such as version A and version B) to determine which version performs better on specific metrics.

[0029] Application development refers to the process of creating and optimizing an APP (Application, application program), including front-end development (user interface design) and back-end development (server-side logic implementation). It involves a variety of technologies and tools, aiming to provide a feature-rich and easy-to-use online experience. It involves technologies such as front-end technology, back-end technology, and user interface design.

[0030] Figure 1 It is a schematic flowchart of the experimental traffic splitting method according to an embodiment of the present disclosure. As Figure 1 shown, the method includes:

[0031] S101, in response to the first request traffic meeting the traffic splitting conditions of the experiment, obtain the information of the first request traffic and the configuration information of the experiment, where the experiment includes multiple candidate experiment versions.

[0032] S102, extract the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment.

[0033] It should be noted that the execution subject of the experimental traffic splitting method of the embodiments of the present disclosure can be a hardware device with data information processing capabilities and / or the necessary software for driving the hardware device to work. Optionally, the execution subject includes the service side, such as including a service server, a service client (such as a web browser, an APP), etc.

[0034] Request traffic refers to the business requests for the service side. For example, the request traffic is triggered based on the operations of the service client or actively triggered by the service client. There are no excessive restrictions on the traffic splitting conditions of the experiment, and different experiments can set different traffic splitting conditions.

[0035] Optionally, the user can open the service client on the terminal device and perform operations such as account login on the page of the service client. Correspondingly, the service client can generate request traffic based on the above operations. Taking the execution entity as the service server as an example, the service client can send the request traffic and the information of the request traffic to the service server.

[0036] It should be noted that the information of the request traffic is not overly restricted. For example, it includes the identifier of the request traffic, the behavior information of the request traffic, the user portrait of the request traffic, the device information of the device from which the request traffic originates, the network environment information of the device from which the request traffic originates, etc. The information of different request traffic may be different.

[0037] Optionally, the identifier of the request traffic includes the user identifier corresponding to the request traffic, such as UID (User Identifier, user unique identifier), account number, etc. The types of UIDs adopted by different service ends may be different.

[0038] Optionally, the identifier of the request traffic includes the number of the request traffic.

[0039] The behavior information of the request traffic refers to the user behavior information corresponding to the request traffic, such as the interaction information between the user corresponding to the request traffic and the service client, such as browsing behavior, search behavior, click behavior, interaction behavior, and other behavior information.

[0040] There is no overly strict restriction on the device from which the request traffic originates. For example, it includes mobile phones, computers, smart home appliances, in-vehicle terminals, etc.

[0041] Optionally, the device information includes device model, operating system type and version number, client type and version number, unique identifier of the device, etc. Among them, the unique identifier includes IMEI (International Mobile Equipment Identity) code, UUID (Universally Unique Identifier), etc.

[0042] Optionally, the network environment information includes network type (such as wireless network, mobile data network), network service provider information, IP (Internet Protocol) address, geographical location information to which the IP address belongs (such as administrative division), etc.

[0043] It should be noted that multiple candidate experiment versions of the same experiment adopt different experiment strategies.

[0044] For example, Experiment 1 includes Version A, Version B, and Version C. Experiment 1 is an experiment on the color of the home page button of an application. In Version A, the button color is set to red; in Version B, the button color is set to green; and in Version C, the button color is set to blue.

[0045] For example, Experiment 2 includes Version D, Version E, and Version F. Experiment 2 is an experiment on the position of the home page button of an application. In Version D, the button is placed on the left; in Version E, the button is placed in the middle; and in Version F, the button is placed on the right.

[0046] It should be noted that the configuration information of the experiment includes the identifier of the experiment and the scenario information of the experiment.

[0047] There are no excessive restrictions on the identifier of the experiment. For example, it includes the name of the experiment, the number of the experiment, etc.

[0048] There are no excessive restrictions on the scenario information of the experiment. For example, it includes the scenario identifier of the experiment (such as the name of the experiment scenario, the scenario number), the scenario category (such as finance, law, vehicle, smart home, entertainment, etc.), the scenario description text, etc. The scenario description text refers to the text used to describe the experiment scenario. For example, the scenario description text can be the experiment on the color of the home page button of a certain video application.

[0049] Optionally, the configuration information of the experiment further includes the configuration information of the candidate experiment versions, the traffic allocation strategy of the experiment, the start time of the experiment, the end time of the experiment, etc.

[0050] Among them, the configuration information of the candidate experiment versions includes the identifier of the candidate experiment versions (such as the name of the candidate experiment version, the version number), the experiment strategy, etc.

[0051] S103. Based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment, determine the target experiment version from multiple candidate experiment versions.

[0052] S104. Allocate the first request traffic to the traffic group corresponding to the target experiment version, and store the splitting result of the first request traffic in the local storage space.

[0053] It should be noted that the target experiment version refers to the candidate experiment version to be tested by the first request traffic, and multiple candidate experiment versions include the target experiment version. There is a corresponding relationship between the candidate experiment versions and the traffic groups, and each traffic group is composed of multiple request traffics.

[0054] In this disclosure, the identifiers of multiple experiments may be different, and the scenario information of multiple experiments may also be different. The request traffic information, experiment identifiers, and experiment scenario information can be comprehensively considered for experiment traffic splitting, so that multiple experiments can be split according to their respective experiment identifiers and scenario information.

[0055] If the identifiers of multiple experiments are the same, the scenario information of multiple experiments can be used to ensure that the traffic splitting results of the same request traffic in different experiments are different.

[0056] In addition, if the scenario information of multiple experiments is the same, the experiment identifiers of multiple experiments can be used to ensure that the traffic splitting results of the same request traffic in different experiments are different.

[0057] It can ensure that the traffic splitting results of the same request traffic in different experiments are different, improve the uniformity and orthogonality of experiment traffic splitting, and further improve the accuracy of experiment results.

[0058] It should be noted that there are no excessive restrictions on the traffic splitting results of request traffic. For example, it can include the configuration information of the traffic group to which the request traffic is allocated (such as the identifier of the traffic group), the configuration information of the target experiment version, the version number of the configuration information of the experiment, etc.

[0059] In this disclosure, the traffic splitting result of the first request traffic is stored in the local storage space. Thus, for some request traffic, there is no need to perform experiment traffic splitting. Only the pre-stored traffic splitting result needs to be obtained from the local storage space, which can improve the experiment traffic splitting efficiency, quickly respond to user requests, optimize the user experience, and also reduce the computing resources occupied by experiment traffic splitting on the service side, contributing to improving the response speed and concurrent processing ability of the service side.

[0060] In addition, it can ensure that the same account is assigned to the same group throughout the experiment, ensuring the consistency of experiment traffic splitting.

[0061] For example, continuing with Experiment 1 as an example, the traffic groups corresponding to the A experiment version, B experiment version, and C experiment version are Traffic Group 1, Traffic Group 2, and Traffic Group 3 respectively. The target experiment versions to be tested for the request traffic within Traffic Group 1 are all the A experiment version, the target experiment versions to be tested for the request traffic within Traffic Group 2 are all the B experiment version, and the target experiment versions to be tested for the request traffic within Traffic Group 3 are all the C experiment version.

[0062] Taking the AB experiment as an example, the traffic group includes an experimental group and a control group. For example, continuing with Experiment 1 as an example, if the color of the home page button of the application is originally red, then the traffic group corresponding to the A experiment version is the control group, and the traffic groups corresponding to the B and C experiment versions are both experimental groups. The control group refers to the traffic group in which the candidate experiment version to be tested is the original version, and the experimental group refers to the traffic group in which the candidate experiment version to be tested is the new version.

[0063] Optionally, based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment, determine the target experiment version from multiple candidate experiment versions, including performing a hashing operation on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment to obtain a target value, and based on the target value, determine the target experiment version from multiple candidate experiment versions.

[0064] In some examples, performing a hashing operation on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment to obtain a target value includes performing a hashing operation on the identifier of the first request traffic, the identifier of the experiment, and the scenario identifier of the experiment to obtain a target value.

[0065] Optionally, based on the target value, determine the target experiment version from multiple candidate experiment versions, including the following possible implementation manners:

[0066] Method 1: Perform a modulo operation on the target value and the total traffic to be allocated for the experiment to obtain a first value. Based on the first value and the traffic quantities to be allocated for each of the multiple candidate experiment versions, determine the target experiment version from the multiple candidate experiment versions.

[0067] Optionally, the total traffic to be allocated for the experiment is N, the value range of the first value is from 0 to N - 1, and N is a positive integer greater than 1.

[0068] It can be understood that the traffic quantities to be allocated for each of the multiple candidate experiment versions may be the same or different. The traffic quantities to be allocated for each of the multiple candidate experiment versions can be obtained by using any experiment traffic splitting method in the related art, and no excessive limitation is made here.

[0069] Optionally, the method further includes determining the traffic quantities to be allocated for each of the multiple candidate experiment versions based on the traffic allocation strategy of the experiment.

[0070] For example, continuing with Experiment 1 as an example, if the traffic allocation strategy of Experiment 1 is that the traffic allocation ratios for Experiment Version A, Experiment Version B, and Experiment Version C are all 1 / 3, and the total traffic to be allocated for Experiment 1 is 150, then the traffic quantities to be allocated for Experiment Version A, Experiment Version B, and Experiment Version C are all 50.

[0071] If the first value corresponding to the first request traffic is from 0 to 49, it can be determined that the target experiment version corresponding to the first request traffic is Experiment Version A, and allocate the first request traffic to the traffic group corresponding to Experiment Version A.

[0072] If the first value corresponding to the first request traffic is from 50 to 99, it can be determined that the target experiment version corresponding to the first request traffic is Experiment Version B, and allocate the first request traffic to the traffic group corresponding to Experiment Version B.

[0073] If the first value corresponding to the first request traffic is from 100 to 149, it can be determined that the target experimental version corresponding to the first request traffic is the C experimental version, and the first request traffic is allocated to the traffic group corresponding to the A experimental version.

[0074] For example, continuing with Experiment 2, if the traffic allocation strategy for Experiment 2 is that the respective traffic allocation ratios for the D experimental version, the E experimental version, and the F experimental version are 10%, 20%, and 70% respectively, and the total traffic to be allocated in Experiment 2 is 100, then the respective traffic quantities to be allocated for the D experimental version, the E experimental version, and the F experimental version are 10, 20, and 70.

[0075] If the first value corresponding to the first request traffic is from 0 to 9, it can be determined that the target experimental version corresponding to the first request traffic is the D experimental version, and the first request traffic is allocated to the traffic group corresponding to the D experimental version.

[0076] If the first value corresponding to the first request traffic is from 10 to 29, it can be determined that the target experimental version corresponding to the first request traffic is the E experimental version, and the first request traffic is allocated to the traffic group corresponding to the E experimental version.

[0077] If the first value corresponding to the first request traffic is from 30 to 99, it can be determined that the target experimental version corresponding to the first request traffic is the F experimental version, and the first request traffic is allocated to the traffic group corresponding to the F experimental version.

[0078] Method 2: Perform a modulo operation on the target value and the number of candidate experimental versions to obtain a second value, and based on the second value, determine the target experimental version from multiple candidate experimental versions.

[0079] Optionally, the number of candidate experimental versions is M, and the value range of the second value is from 0 to M - 1.

[0080] Optionally, determining the target experimental version from multiple candidate experimental versions based on the second value includes, in response to the second value being Q and Q being non-zero, taking the Qth candidate experimental version as the target experimental version, and in response to the second value being zero, taking the Mth candidate experimental version as the target experimental version.

[0081] For example, continuing with Experiment 2, if the traffic allocation strategy for Experiment 2 is that the respective traffic allocation ratios for the D experimental version, the E experimental version, and the F experimental version are 10%, 20%, and 70% respectively, and the total traffic to be allocated in Experiment 2 is 100, then the respective traffic quantities to be allocated for the D experimental version, the E experimental version, and the F experimental version are 10, 20, and 70.

[0082] If the second value corresponding to the first request traffic is 1, it can be determined that the target experimental version corresponding to the first request traffic is the D experimental version, and the first request traffic is allocated to the traffic group corresponding to the D experimental version.

[0083] If the second value corresponding to the first request traffic is 2, it can be determined that the target experimental version corresponding to the first request traffic is the E experimental version, and the first request traffic is allocated to the traffic group corresponding to the E experimental version.

[0084] If the second value corresponding to the first request traffic is 0, it can be determined that the target experimental version corresponding to the first request traffic is the F experimental version, and the first request traffic is allocated to the traffic group corresponding to the F experimental version.

[0085] Optionally, the business side deploys an SDK (Software Development Kit) of the experimental platform, and at least one of the steps S101 - S104 can be executed by the SDK of the experimental platform.

[0086] The experimental traffic splitting method proposed by the present disclosure, in response to the first request traffic meeting the traffic splitting conditions of the experiment, obtains the information of the first request traffic and the configuration information of the experiment. Among them, the experiment includes multiple candidate experimental versions. The identifier of the experiment and the scenario information of the experiment are extracted from the configuration information of the experiment. Based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment, the target experimental version is determined from the multiple candidate experimental versions. The first request traffic is allocated to the traffic group corresponding to the target experimental version, and the traffic splitting result of the first request traffic is stored in the local storage space. Thus, it is possible to comprehensively consider the information of the request traffic, the identifier of the experiment, and the scenario information of the experiment for experimental traffic splitting. As a result, multiple experiments can be split according to their respective experimental identifiers and respective scenario information, ensuring that the traffic splitting results of the same request traffic in different experiments are different, which helps to improve the uniformity and orthogonality of experimental traffic splitting, and further improves the accuracy of experimental results, and is applicable to experimental traffic splitting scenarios in fields such as finance, law, vehicles, smart homes, and entertainment.

[0087] In addition, storing the traffic splitting result of the first request traffic in the local storage space can improve the experimental traffic splitting efficiency, quickly respond to user requests, optimize the user experience, and also reduce the computing resources occupied by experimental traffic splitting on the business side, thereby helping to improve the response speed and concurrent processing ability of the business side.

[0088] Figure 2 For the flowchart of the experimental traffic splitting method according to another embodiment of the present disclosure, as Figure 2 shown, the method includes:

[0089] S201. In response to the first request traffic meeting the shunting conditions of the experiment, obtain the information of the first request traffic and the configuration information of the experiment, where the experiment includes multiple candidate experiment versions.

[0090] S202. Extract the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment.

[0091] For the relevant content of steps S201 - S202, reference can be made to the above embodiments and will not be elaborated here.

[0092] S203. Based on the scenario information of the experiment, determine the target shunting strategy of the experiment.

[0093] In the present disclosure, the target shunting strategy of the experiment can be determined considering the scenario information of the experiment. Multiple experiments can determine their respective target shunting strategies based on their respective scenario information, improving the flexibility of the target shunting strategy.

[0094] It can be understood that the target shunting strategies of different experiments may be different. There are no excessive limitations on the shunting strategy. For example, it may include the traffic allocation ratio, shunting function, etc.

[0095] In the present disclosure, based on the scenario information of the experiment, determining the target shunting strategy of the experiment includes the following possible implementation manners:

[0096] Way 1. Obtain the mapping relationship between the candidate scenario information and the candidate shunting strategy, obtain the similarity between the candidate scenario information and the scenario information of the experiment, and use the candidate scenario information with the largest similarity as the target scenario information. Obtain the candidate shunting strategy that has a mapping relationship with the target scenario information as the target shunting strategy.

[0097] In this embodiment, the mapping relationship between the candidate scenario information and the candidate shunting strategy can be established in advance, and the candidate shunting strategy mapped by the candidate scenario information most similar to the scenario information of the experiment is used as the target shunting strategy.

[0098] For example, there is a mapping relationship between candidate scenario information 1 and candidate shunting strategy 1, a mapping relationship between candidate scenario information 2 and candidate shunting strategy 2, and a mapping relationship between candidate scenario information 3 and candidate shunting strategy 3.

[0099] Continuing with Experiment 1 as an example, the similarity between candidate scenario information 1 and the scenario information of Experiment 1 is 50%, the similarity between candidate scenario information 2 and the scenario information of Experiment 1 is 80%, and the similarity between candidate scenario information 3 and the scenario information of Experiment 1 is 40%.

[0100] Then, the candidate scenario information 2 with the largest similarity can be used as the target scenario information, and the candidate shunting strategy 2 can be used as the target shunting strategy of Experiment 1.

[0101] Method 2: Determine the scenario category of the experiment based on the scenario information of the experiment. In response to the scenario category of the experiment being the first category, determine that the traffic allocation ratio of the experimental group of the experiment is in the first interval.

[0102] In response to the scenario category of the experiment being the second category, determine that the traffic allocation ratio of the experimental group of the experiment is in the second interval.

[0103] In response to the scenario category of the experiment being the third category, determine that the traffic allocation ratio of the experimental group of the experiment is in the third interval.

[0104] Wherein, the upper limit value of the first interval is less than or equal to the lower limit value of the second interval, and the upper limit value of the second interval is less than or equal to the lower limit value of the third interval.

[0105] Thus, considering the scenario category of the experiment, the traffic allocation ratio of the experimental group of the experiment can be determined, improving the flexibility of the traffic allocation ratio of the experimental group of the experiment.

[0106] There are no excessive limitations on the first to third categories. For example, the first category includes finance and law, the second category includes vehicles and smart homes, and the third category includes entertainment, such as games, videos, music, social chatting, etc.

[0107] There are no excessive limitations on the first to third intervals. For example, the first interval is 5% to 10%, the second interval is 10% to 30%, and the third interval is 30% to 50%.

[0108] In the present disclosure, if the scenario category of the experiment is the first category, the traffic allocation ratio of the experimental group of the experiment is the smallest. For example, in scenarios such as finance and law, the tolerance for mistakes is relatively low. Setting a smaller traffic allocation ratio for the experimental group helps reduce the experimental risk of the new version.

[0109] If the scenario category of the experiment is the second category, the traffic allocation ratio of the experimental group of the experiment is moderate. For example, in scenarios such as vehicles and smart homes, setting a moderate traffic allocation ratio for the experimental group can quickly obtain user feedback on the new version while ensuring a relatively low experimental risk for the new version.

[0110] If the scenario category of the experiment is the third category, the traffic allocation ratio of the experimental group of the experiment is the largest. For example, applications in the entertainment scenario need to be iterated frequently. Setting a larger traffic allocation ratio for the experimental group can quickly obtain user feedback on the new version.

[0111] Optionally, determining the scenario category of the experiment based on the scenario information of the experiment includes performing semantic analysis on the scenario information of the experiment to obtain the scenario category of the experiment, or extracting the scenario category of the experiment from the configuration information of the experiment.

[0112] It should be noted that in this embodiment, only three gears of the first to the third intervals are set for the traffic allocation ratio of the experimental group, which is only an example of the traffic allocation ratio of the experimental group in the embodiments of the present disclosure, and does not limit the traffic allocation ratio of the experimental group in the embodiments of the present disclosure. For example, four gears of the first to the fourth intervals can be set for the traffic allocation ratio of the experimental group, and in response to the experimental scenario category being the fourth category, it is determined that the traffic allocation ratio of the experimental group of the experiment is in the fourth interval, and the upper limit value of the third interval is less than or equal to the lower limit value of the fourth interval.

[0113] Method 3: Determine the experimental scenario category based on the experimental scenario information, and based on the experimental scenario category, determine the weights of the N attributes of the candidate shunt function respectively, where N is an integer greater than or equal to 1. Based on the N attributes of the candidate shunt function and the weights of the N attributes respectively, obtain the score of the candidate shunt function, and based on the score, determine the target shunt function of the experiment from multiple candidate shunt functions.

[0114] Thus, considering the experimental scenario category, the weights of the N attributes of the candidate shunt function can be determined to calculate the score of the candidate shunt function, and then the target shunt function of the experiment can be determined from multiple candidate shunt functions, improving the flexibility of the target shunt function.

[0115] There are no excessive limitations on the shunt function. For example, it can include a hash function. There are no excessive limitations on the N attributes. For example, it can include the security level and the calculation speed.

[0116] Optionally, the N attributes include the security level and the calculation speed. In the case where the experimental scenario category is the first category, the weight of the security level is higher than the weight of the calculation speed.

[0117] In the case where the experimental scenario category is the second category or the third category, the weight of the security level is lower than the weight of the calculation speed.

[0118] Thus, considering the experimental scenario category, the weights of the security level and the calculation speed can be determined.

[0119] In the present disclosure, if the experimental scenario category is the first category, such as in scenarios like finance and law, the weight of the security level is higher than the weight of the calculation speed, which helps to enhance data security and avoid data leakage.

[0120] If the experimental scenario category is the second category, such as in scenarios like vehicles and smart homes, or if the experimental scenario category is the third category, such as in the entertainment scenario, the weight of the security level is lower than the weight of the calculation speed, which helps to improve the experimental shunt efficiency to meet the high-frequency iteration requirements.

[0121] Optionally, based on the experimental scenario category, determine the weights of the N attributes of the candidate shunt function respectively, including obtaining the mapping relationship between the candidate scenario category and the candidate weights of the attributes, and obtaining the candidate weights of the attributes that have a mapping relationship with the experimental scenario category as the final weights of the attributes.

[0122] Optionally, based on the N attributes of the candidate shunt function and the weights of the N attributes respectively, obtain the score of the candidate shunt function, including performing weighted summation on the N attributes of the candidate shunt function based on the weights of the N attributes respectively to obtain the score of the candidate shunt function.

[0123] Optionally, based on the score, determine the target shunt function of the experiment from multiple candidate shunt functions, including taking the candidate shunt function with the maximum score as the target shunt function.

[0124] S204. Process the information of the first request traffic and the identifier of the experiment according to the target shunt strategy to determine the target experiment version from multiple candidate experiment versions.

[0125] Optionally, process the information of the first request traffic and the identifier of the experiment according to the target shunt strategy to determine the target experiment version from multiple candidate experiment versions, including performing a hash operation on the information of the first request traffic and the identifier of the experiment according to the target hash function to determine the target experiment version from multiple candidate experiment versions.

[0126] It should be noted that the relevant content for determining the target experiment version through the hash operation can refer to the relevant content of the above embodiments and will not be elaborated here.

[0127] S205. Allocate the first request traffic to the traffic group corresponding to the target experiment version and store the shunt result of the first request traffic in the local storage space.

[0128] The relevant content of step S205 can be referred to the above embodiments and will not be elaborated here.

[0129] The experimental shunt method proposed by the present disclosure determines the target shunt strategy of the experiment based on the scenario information of the experiment, and processes the information of the first request traffic and the identifier of the experiment according to the target shunt strategy to determine the target experiment version from multiple candidate experiment versions. Thus, the target shunt strategy of the experiment can be determined considering the scenario information of the experiment, and multiple experiments can determine their respective target shunt strategies according to their respective scenario information, improving the flexibility of the target shunt strategy, and the experimental shunt can be achieved by processing the information of the first request traffic and the identifier of the experiment according to the target shunt strategy.

[0130] Figure 3 It is a schematic flowchart of the experimental shunt method according to another embodiment of the present disclosure, as Figure 3As shown, the method includes:

[0131] S301, in response to the first request traffic meeting the shunting conditions of the experiment, obtain the information of the first request traffic and the configuration information of the experiment, where the experiment includes multiple candidate experiment versions.

[0132] S302, extract the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment.

[0133] For the relevant content of steps S301 - S302, reference can be made to the above - mentioned embodiments, which will not be elaborated here.

[0134] S303, obtain the target shunting strategy of the experiment.

[0135] S304, according to the target shunting strategy, process the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment to determine the target experiment version from multiple candidate experiment versions.

[0136] It can be understood that the target shunting strategies of different experiments may be different.

[0137] Optionally, processing the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment according to the target shunting strategy to determine the target experiment version from multiple candidate experiment versions includes performing a hash operation on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment according to the target hash function to determine the target experiment version from multiple candidate experiment versions.

[0138] It should be noted that for the relevant content of the hash operation to determine the target experiment version, reference can be made to the relevant content of the above - mentioned embodiments, which will not be elaborated here.

[0139] S305, allocate the first request traffic to the traffic group corresponding to the target experiment version, and store the shunting result of the first request traffic in the local storage space.

[0140] For the relevant content of step S305, reference can be made to the above - mentioned embodiments, which will not be elaborated here.

[0141] The experiment shunting method proposed by the present disclosure obtains the target shunting strategy of the experiment, and according to the target shunting strategy, processes the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment to determine the target experiment version from multiple candidate experiment versions. Thus, the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment can be processed according to the target shunting strategy to perform experiment shunting.

[0142] Figure 4 It is a schematic flowchart of the experiment shunting method according to another embodiment of the present disclosure. As Figure 4 shown, the method includes:

[0143] S401. In response to the first request traffic meeting the shunting conditions of the experiment, obtain the information of the first request traffic and the configuration information of the experiment, where the experiment includes multiple candidate experiment versions.

[0144] S402. Extract the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment.

[0145] For the relevant content of steps S401 - S402, reference can be made to the above embodiments and will not be elaborated here.

[0146] S402. Obtain the random number corresponding to the experiment.

[0147] It can be understood that different experiments may correspond to different random numbers. The random number includes a salt value.

[0148] S404. Based on the information of the first request traffic, the identifier of the experiment, the scenario information of the experiment, and the random number corresponding to the experiment, determine the target experiment version from multiple candidate experiment versions.

[0149] In the present disclosure, the random numbers corresponding to multiple experiments may be different. The experiment shunting can be performed by comprehensively considering the information of the request traffic, the identifier of the experiment, the scenario information of the experiment, and the random number corresponding to the experiment. Thus, multiple experiments can be shunted according to their respective experiment identifiers, respective scenario information, and respective corresponding random numbers.

[0150] If the identifiers and scenario information of multiple experiments are the same, the respective random numbers corresponding to the multiple experiments can be used to ensure that the shunting results of the same request traffic in different experiments are different.

[0151] It can be ensured that the shunting results of the same request traffic in different experiments are different, improving the uniformity and orthogonality of experiment shunting, and further improving the accuracy of experiment results.

[0152] Optionally, based on the information of the first request traffic, the identifier of the experiment, the scenario information of the experiment, and the random number corresponding to the experiment, determining the target experiment version from multiple candidate experiment versions includes performing a hash operation on the identifier of the request traffic, the identifier of the experiment, and the random number corresponding to the experiment to obtain a target value, and based on the target value, determining the target experiment version from multiple candidate experiment versions. Thus, the hash operation can be performed on the identifier of the request traffic, the identifier of the experiment, and the random number corresponding to the experiment to achieve experiment shunting.

[0153] It should be noted that for the relevant content of determining the target experiment version from multiple candidate experiment versions based on the target value, reference can be made to the above embodiments and will not be elaborated here.

[0154] Optionally, based on the information of the first request traffic, the identifier of the experiment, the scenario information of the experiment, and the random number corresponding to the experiment, determine the target experiment version from multiple candidate experiment versions, including determining the target traffic splitting strategy of the experiment based on the scenario information of the experiment, and processing the information of the first request traffic, the identifier of the experiment, and the random number corresponding to the experiment according to the target traffic splitting strategy, so as to determine the target experiment version from multiple candidate experiment versions.

[0155] Optionally, based on the information of the first request traffic, the identifier of the experiment, the scenario information of the experiment, and the random number corresponding to the experiment, determine the target experiment version from multiple candidate experiment versions, including obtaining the target traffic splitting strategy of the experiment, and processing the information of the first request traffic, the identifier of the experiment, the scenario information of the experiment, and the random number corresponding to the experiment according to the target traffic splitting strategy, so as to determine the target experiment version from multiple candidate experiment versions.

[0156] S405, allocate the first request traffic to the traffic group corresponding to the target experiment version, and store the traffic splitting result of the first request traffic in the local storage space.

[0157] For the relevant content of step S405, refer to the above embodiments, and details are not described here again.

[0158] For the experiment traffic splitting method proposed by the present disclosure, obtain the random number corresponding to the experiment, and determine the target experiment version from multiple candidate experiment versions based on the information of the first request traffic, the identifier of the experiment, the scenario information of the experiment, and the random number corresponding to the experiment. Thus, it is possible to comprehensively consider the information of the request traffic, the identifier of the experiment, the scenario information of the experiment, and the random number corresponding to the experiment for experiment traffic splitting. As a result, multiple experiments can be split according to their respective experiment identifiers, respective scenario information, and respective corresponding random numbers, which can ensure that the traffic splitting results of the same request traffic in different experiments are different, improve the uniformity and orthogonality of the experiment traffic splitting, and further improve the accuracy of the experiment results.

[0159] Figure 5 It is a schematic flowchart of the experiment traffic splitting method according to another embodiment of the present disclosure. As Figure 5 shown, the method includes:

[0160] S501, in response to the first request traffic meeting the traffic splitting condition of the experiment, obtain the information of the first request traffic and the configuration information of the experiment, where the experiment includes multiple candidate experiment versions.

[0161] S502, extract the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment.

[0162] S503, based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment, determine the target experiment version from multiple candidate experiment versions.

[0163] S504, allocate the first request traffic to the traffic group corresponding to the target experimental version.

[0164] For the relevant content of steps S501 - S504, reference can be made to the above embodiments, which will not be elaborated here.

[0165] S505, perform semantic analysis on the configuration information of the experiment to obtain the object category of the experiment.

[0166] S506, determine the target embedding point strategy for the first request traffic based on the object category of the experiment.

[0167] In this disclosure, the target embedding point strategy can be determined considering the object category of the experiment. Multiple experiments can determine their respective target embedding point strategies according to their respective scenario information, improving the flexibility of the target embedding point strategy.

[0168] It should be noted that there are no excessive limitations on the object category, such as including text, images, voice, etc. There are no excessive limitations on the embedding point strategy, such as including the category of embedding point information, the amount of collected embedding point information, the collection time period of embedding point information, etc. There are no excessive limitations on the embedding point information, such as including the behavior information of the request traffic, the metrics of the request traffic, etc.

[0169] Optionally, the category of the embedding point information of the first request traffic includes at least one of the general metrics and private metrics. It can be understood that the private metric is a metric private to a certain business end, improving the personalization of metric calculation.

[0170] The general metric is a metric shared by multiple business ends, which can be uniformly defined by the experimental platform, for example. The private metric is a metric private to a certain business end and can be personalized defined by a certain business end. There are no excessive limitations on the general metrics and private metrics. For example, the general metrics include click - through rate, number of impressions, PV, UV, etc.

[0171] In this disclosure, determining the target embedding point strategy for the first request traffic based on the object category of the experiment includes the following possible implementation manners:

[0172] Method 1: Obtain the mapping relationship between the candidate object category and the candidate embedding point strategy, and obtain the candidate embedding point strategy that has a mapping relationship with the object category of the experiment as the target embedding point strategy.

[0173] In this embodiment, the mapping relationship between the candidate object category and the candidate embedding point strategy can be established in advance, and the candidate embedding point strategy mapped by the object category of the experiment is used as the target embedding point strategy.

[0174] Method 2: In response to the object category of the experiment being text, determine that the category of the embedding point information of the first request traffic includes M categories.

[0175] In response to the object category of the experiment being an image, it is determined that the buried point information category of the first request traffic includes P categories.

[0176] In response to the object category of the experiment being speech, it is determined that the buried point information category of the first request traffic includes Q categories.

[0177] Wherein, M, P, and Q are all integers greater than or equal to 1. The P categories and the Q categories each include at least M categories, or P and Q are both greater than M, and there is partial overlap between the P categories and the M categories, and there is partial overlap between the Q categories and the M categories.

[0178] Thus, considering the object category of the experiment, the buried point information category of the request traffic can be determined. The buried point information categories corresponding to images and speech include the buried point information category corresponding to text.

[0179] Or, compared with the buried point information category corresponding to text, the buried point information categories corresponding to images and speech are more, and there is partial overlap between the buried point information category corresponding to images and the buried point information category corresponding to text, and there is partial overlap between the buried point information category corresponding to speech and the buried point information category corresponding to text.

[0180] There are no excessive restrictions on the M categories, the P categories, and the Q categories.

[0181] For example, the M categories, the P categories, and the Q categories all include click-through rate, display times, PV (Page Views), UV (Unique Visitors), etc.

[0182] The M categories include the scrolling position of the mouse, font preference, etc.

[0183] The P categories include image zooming behavior, image downloading behavior, etc.

[0184] The Q categories include behaviors such as volume adjustment, play speed adjustment, pause play, resume play, etc.

[0185] S507. Obtain the buried point information of the first request traffic according to the target buried point strategy.

[0186] It should be noted that the relevant content of obtaining the buried point information of the first request traffic according to the target buried point strategy can be implemented by any data buried point method in the related technology, and there are no excessive restrictions here.

[0187] The experimental traffic splitting method proposed by the present disclosure performs semantic analysis on the configuration information of the experiment to obtain the object category of the experiment. Based on the object category of the experiment, the target embedding strategy for the first request traffic is determined, and according to the target embedding strategy, the embedding information of the first request traffic is obtained. Thus, considering the object category of the experiment, the target embedding strategy can be determined, and multiple experiments can determine their respective target embedding strategies according to their respective scenario information, improving the flexibility of the target embedding strategy and enabling data embedding according to the target embedding strategy.

[0188] Figure 6 It is a schematic flowchart of the experimental traffic splitting method according to another embodiment of the present disclosure. As Figure 6 shown, the method includes:

[0189] S601, in response to the first request traffic meeting the traffic splitting condition of the experiment, obtain the information of the first request traffic and the configuration information of the experiment, where the experiment includes multiple candidate experiment versions.

[0190] S602, extract the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment.

[0191] S603, based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment, determine the target experiment version from multiple candidate experiment versions.

[0192] S604, allocate the first request traffic to the traffic group corresponding to the target experiment version, and store the traffic splitting result of the first request traffic in the local storage space.

[0193] For the relevant content of steps S601 - S604, reference can be made to the above embodiments and will not be elaborated here.

[0194] S605, in response to the second request traffic meeting the traffic splitting condition of the experiment.

[0195] S606, when the second request traffic meets at least one of the conditions that the identifier of the second request traffic is the same as the identifier of the first request traffic and the second request traffic and the first request traffic come from the same device, use the traffic splitting result of the first request traffic in the local storage space as the traffic splitting result of the second request traffic.

[0196] In the present disclosure, when the second request traffic meets at least one of the conditions that the identifier of the second request traffic is the same as the identifier of the first request traffic and the second request traffic and the first request traffic come from the same device, there is no need to perform experimental traffic splitting on the second request traffic. Using the traffic splitting result of the first request traffic in the local storage space as the traffic splitting result of the second request traffic can improve the experimental traffic splitting efficiency, quickly respond to user requests, optimize the user experience, and also reduce the computing resources occupied by experimental traffic splitting on the service side, helping to improve the response speed and concurrent processing ability of the service side.

[0197] In addition, it can be ensured that the same account is assigned to the same group throughout the experiment, ensuring the consistency of experimental traffic splitting.

[0198] Optionally, before using the traffic splitting result of the first request traffic in the local storage space as the traffic splitting result of the second request traffic, it also includes determining that the configuration information of the experiment has not changed. Thus, after determining that the configuration information of the experiment has not changed, the pre-stored traffic splitting result can be obtained from the local storage space, ensuring the accuracy of experimental traffic splitting.

[0199] It can be understood that when the configuration information of the experiment has not changed, the locally stored traffic splitting result is still applicable to the configuration information of the experiment. The locally stored traffic splitting result can be retained to avoid frequently updating the locally stored traffic splitting result. It can be ensured that the same account is assigned to the same group throughout the experiment, ensuring the consistency of experimental traffic splitting.

[0200] In the related art, when the information of the request traffic is abnormal, it will cause the traffic splitting result of the request traffic to be frequently updated. For example, when an account is abnormal, the traffic splitting result re-obtained by the account may be inconsistent with the original traffic splitting result of the account, resulting in frequent updates of the traffic splitting result of the request traffic.

[0201] However, in this solution, when the information of the request traffic is abnormal, if the configuration information of the experiment has not changed, the locally stored traffic splitting result is retained. At this time, the experimental traffic splitting will not be re-performed, or even if the experimental traffic splitting is re-performed, the re-obtained traffic splitting result will not be adopted, avoiding frequent updates of the locally stored traffic splitting result when the information of the request traffic is abnormal. It can be ensured that when the information of the request traffic is abnormal, the same account is assigned to the same group throughout the experiment, ensuring the consistency of experimental traffic splitting.

[0202] It can be understood that when the configuration information of the experiment changes, the locally pre-stored traffic splitting result may not be applicable to the latest configuration information of the experiment.

[0203] Optionally, when the second request traffic meets at least one condition, the method further includes obtaining the shunt result of the second request traffic in response to a change in the configuration information of the experiment, and replacing the shunt result of the first request traffic in the local storage space with the shunt result of the second request traffic. Thus, when the second request traffic meets at least one of the conditions that the identifier of the second request traffic is the same as the identifier of the first request traffic and the second request traffic and the first request traffic come from the same device, if the configuration information of the experiment changes, it is necessary to perform experimental shunting on the second request traffic, and replace the shunt result of the first request traffic in the local storage space with the shunt result of the second request traffic, that is, update the shunt result stored locally, improving the timeliness and accuracy of the shunt result stored locally, and further improving the timeliness and accuracy of the experimental result.

[0204] It should be noted that for the relevant content of obtaining the shunt result of the second request traffic, reference can be made to the relevant content of obtaining the shunt result of the first request traffic in the above embodiments, which will not be elaborated here.

[0205] In the present disclosure, identifying whether the configuration information of the experiment has changed includes the following possible implementation manners:

[0206] Method 1: Determine that the configuration information of the experiment has changed in response to a change in at least one of the information of the identifier of the experiment, the configuration information of the candidate experiment version, and the traffic allocation strategy of the experiment.

[0207] For example, continuing with Experiment 1 as an example, if the target experiment version to be tested for request traffic X is Version A of the experiment, the identifier of Version A of the experiment and the experimental strategy of Version A of the experiment can be added to the shunt result of request traffic X, and the shunt result of request traffic X is stored in the local storage space.

[0208] Determine that the configuration information of Experiment 1 has changed in response to a change in at least one of the information of the identifier of Experiment 1, the configuration information of Version A of the experiment, the configuration information of Version B of the experiment, the configuration information of Version C of the experiment, and the traffic allocation strategy of Experiment 1.

[0209] In response to a change in the configuration information of Experiment 1, re-obtain the shunt result of request traffic X, and replace the original shunt result of request traffic X in the local storage space with the re-obtained shunt result of request traffic X.

[0210] Method 2: Determine that the configuration information of the experiment has changed in response to a change in the version number of the configuration information of the experiment, and determine that the configuration information of the experiment has not changed in response to no change in the version number of the configuration information of the experiment.

[0211] Thus, it is possible to identify whether the configuration information of the experiment has changed by identifying whether the version number of the configuration information of the experiment has changed.

[0212] Optionally, the method further includes receiving the version number of the configuration information of the experiment sent by the experimental platform. Thus, the version number of the configuration information of the experiment can be managed by the experimental platform, and only the corresponding interface needs to be developed on the business side to receive the version number of the configuration information of the experiment sent by the experimental platform.

[0213] In addition, when the version number of the configuration information of the experiment is dynamically adjusted, the business side only needs to continue to receive the adjusted version number of the configuration information of the experiment through the interface, without adjusting the business side code, reducing the development cost and development time of the business side, and thus reducing the experiment cost and experiment time.

[0214] For example, the configuration information of the experiment and the version number of the configuration information of the experiment are transmitted bundled.

[0215] For example, the configuration information of the experiment is transmitted according to the transmission method of the pass-through parameters. Thus, the version number of the configuration information of the experiment can be transmitted according to the transmission method of the pass-through parameters, which can ensure the integrity and originality of the version number of the configuration information of the experiment.

[0216] For example, continuing with Experiment 1 as an example, during the process of splitting the request traffic X for Experiment 1, receive the configuration information P1 of Experiment 1 and the version number 1.0 of the configuration information P1 of Experiment 1 sent by the experimental platform, extract the identifier button color of Experiment 1 and the scenario information of Experiment 1 from the configuration information P1 of Experiment 1, and determine the target experiment version from the A experiment version, B experiment version, and C experiment version based on the identifier of the request traffic X, the identifier button color of Experiment 1, and the scenario information of Experiment 1.

[0217] If the target experiment version to be tested for the request traffic X is the A experiment version, add the configuration information of the A experiment version to the split result of the request traffic X, and store the split result of the request traffic X in the local storage space.

[0218] During the process of splitting the request traffic Y for Experiment 1, receive the configuration information P2 of Experiment 1 and the version number 2.0 of the configuration information P2 of Experiment 1 sent by the experimental platform, and extract the identifier buttoncolor of Experiment 1 and the scenario information of Experiment 1 from the configuration information P2 of Experiment 1.

[0219] When the request traffic Y satisfies at least one of the conditions that the identifier of the request traffic Y is consistent with the identifier of the request traffic X and the request traffic Y and the request traffic X come from the same device, if the version number of the configuration information of Experiment 1 has not changed, use the split result of the request traffic X in the local storage space as the split result of the request traffic Y.

[0220] Alternatively, if the version number of the configuration information of Experiment 1 changes, based on the identifier of the request traffic Y, the identifier button color of Experiment 1, and the scenario information of Experiment 1, determine the target experiment version from the A experiment version, B experiment version, and C experiment version.

[0221] If the target experiment version to be tested for the request traffic Y is the C experiment version, add the configuration information of the C experiment version to the shunt result of the request traffic X, and replace the shunt result of the request traffic X in the local storage space with the shunt result of the request traffic Y. It should be noted that in this embodiment, the request traffic X is the first request traffic, and the request traffic Y is the second request traffic.

[0222] The experiment shunt method proposed by the present disclosure, in response to the second request traffic satisfying the shunt condition of the experiment, when at least one of the conditions that the identifier of the second request traffic is consistent with the identifier of the first request traffic and the second request traffic and the first request traffic originate from the same device is satisfied, use the shunt result of the first request traffic in the local storage space as the shunt result of the second request traffic. Thus, when the second request traffic satisfies the preset condition, there is no need to perform experiment shunting on the second request traffic, and using the shunt result of the first request traffic in the local storage space as the shunt result of the second request traffic can improve the experiment shunting efficiency, quickly respond to user requests, optimize the user experience, and also reduce the computing resources occupied by experiment shunting on the service side, which helps to improve the response speed and concurrent processing ability of the service side. In addition, it can ensure that the same account is assigned to the same group throughout the experiment, ensuring the consistency of experiment shunting.

[0223] Based on any of the above embodiments, obtain the configuration information of the experiment, including receiving the configuration information of the experiment sent by the experiment platform, where the configuration information of the experiment is transmitted according to the transmission method of the pass-through parameters. Thus, the configuration information of the experiment can be transmitted according to the transmission method of the pass-through parameters, ensuring the integrity and originality of the configuration information of the experiment.

[0224] In addition, by managing the configuration information of the experiment through the experiment platform, only need to develop corresponding interfaces on the service side to receive the configuration information of the experiment sent by the experiment platform.

[0225] In addition, when the configuration information of the experiment is dynamically adjusted, the service side only needs to continue to receive the adjusted configuration information of the experiment through the interface, without adjusting the service side code, reducing the development cost and development time of the service side, and thus reducing the experiment cost and experiment time.

[0226] Figure 7 For the flow diagram of the experiment shunt method of another embodiment of the present disclosure, as Figure 7 shown, the method includes:

[0227] S701. In response to the first request traffic meeting the shunting conditions of the experiment, obtain the information of the first request traffic and the configuration information of the experiment, where the experiment includes multiple candidate experiment versions.

[0228] S702. Extract the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment.

[0229] S703. Based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment, determine the target experiment version from multiple candidate experiment versions.

[0230] S704. Allocate the first request traffic to the traffic group corresponding to the target experiment version.

[0231] For the relevant content of steps S701 - S704, reference can be made to the above embodiments and will not be elaborated here.

[0232] S705. Obtain the buried point information of the first request traffic.

[0233] It should be noted that to obtain the buried point information of the first request traffic, any data buried point method in the related technology can be used to implement it, and no excessive limitation is made here.

[0234] Optionally, obtaining the buried point information of the first request traffic includes obtaining the behavior information of the first request traffic, and based on the behavior information of the first request traffic, obtaining at least one of the general metrics and private metrics of the first request traffic as the buried point information of the first request traffic. Thus, considering the behavior information of the first request traffic, at least one of the general metrics and private metrics of the first request traffic can be obtained as the buried point information of the first request traffic. The private metric is a metric private to a certain business end, which improves the personalization of metric calculation. In addition, the general metrics and / or private metrics of the request traffic can be synchronized to the experiment platform so that the experiment platform can perform statistical processing on the general metrics and / or private metrics of the request traffic, which helps to improve the comprehensiveness and accuracy of the experiment results.

[0235] S706. Associate the buried point information of the first request traffic with the shunting result of the first request traffic to obtain the reporting information of the first request traffic.

[0236] S707. Send the reporting information of the first request traffic to the experiment platform.

[0237] It should be noted that to associate the buried point information of the first request traffic with the shunting result of the first request traffic, any data association method in the related technology can be used to implement it, and no excessive limitation is made here. For example, a mapping relationship, a corresponding relationship, etc. can be established between the buried point information of the first request traffic and the shunting result of the first request traffic.

[0238] For example, the buried point information of request traffic X and the shunt result of request traffic X can be associated to obtain the reported information of request traffic X, and the reported information of request traffic X is sent to the experimental platform.

[0239] For example, the buried point information of request traffic Y and the shunt result of request traffic Y can be associated to obtain the reported information of request traffic Y, and the reported information of request traffic Y is sent to the experimental platform.

[0240] Optionally, the reported information of the first request traffic is reported in a unified format.

[0241] Optionally, the SDK of the experimental platform is deployed on the service side, and at least one of steps S705 - S707 can be executed by the SDK of the experimental platform.

[0242] The experimental shunt method proposed by the present disclosure obtains the buried point information of the first request traffic, associates the buried point information of the first request traffic with the shunt result of the first request traffic to obtain the reported information of the first request traffic, and sends the reported information of the first request traffic to the experimental platform. Thus, the buried point information of the request traffic and the shunt result of the request traffic can be automatically associated, without the need for manual association by the user, which simplifies the user operation, and also does not require the experimental platform to perform the association, simplifies the processing logic of the experimental platform, and improves the experimental efficiency.

[0243] Based on any of the above embodiments, the method further includes, after the first request traffic satisfies the shunt condition of the experiment, sending the first indication information to the experimental platform, where the first indication information is used to indicate that the first request traffic satisfies the shunt condition of the experiment. Thus, the service side can notify the experimental platform that the request traffic satisfies the shunt condition of the experiment, so that the experimental platform can know which request traffic satisfies the shunt condition of the experiment.

[0244] Figure 8 It is a schematic flowchart of the experimental shunt method according to another embodiment of the present disclosure, as Figure 8 shown, the method includes:

[0245] S801, adding the identifier of the experiment and the scenario information of the experiment to the configuration information of the experiment, where the experiment includes multiple candidate experiment versions, and the identifier of the experiment and the scenario information of the experiment are used to determine the target experiment version to be tested for the first request traffic from multiple candidate experiment versions.

[0246] S802, sending the configuration information of the experiment to the service side according to the transmission method of the pass - through parameter.

[0247] It should be noted that the execution subject of the experimental traffic splitting method in the embodiments of the present disclosure can be a hardware device with data information processing capabilities and / or the necessary software for driving the hardware device to work. Optionally, the execution subject includes an experimental platform. For the relevant content of steps S801 - S802, reference can be made to the above embodiments and will not be elaborated here.

[0248] Optionally, the method further includes sending the version number of the configuration information of the experiment to the service side to inform the service side of the version number of the configuration information of the experiment.

[0249] Optionally, after sending the configuration information of the experiment to the service side, it further includes receiving the reporting information of the first requested traffic sent by the service side, and performing statistical processing on the reporting information of the first requested traffic to obtain the experimental result of the experiment when receiving the first indication information sent by the service side, where the first indication information is used to indicate that the first requested traffic meets the traffic splitting condition of the experiment. Thus, compared with the related art where information reporting and information statistics are mostly carried out by two independent systems, there may be problems such as inconsistent information, duplicate development of indicators, and duplicate calculations between the two systems. In the solution of this embodiment, the experimental platform integrates the functions of information reporting and information statistics, can perform statistical processing on the information reported by the service side, ensures the consistency of the information statistical caliber and processing logic, and the information consistency can further improve the traceability of the information, and at the same time further reduces the usage cost of the service side.

[0250] In addition, when receiving the first indication information sent by the service side, that is, when the first requested traffic meets the traffic splitting condition of the experiment, statistical processing can be performed on the reporting information of the first requested traffic to ensure that only the requested traffic that enters the group is included in the experimental statistics during the experimental statistics stage, improving the accuracy of the experimental result.

[0251] Optionally, performing statistical processing on the reporting information of the first requested traffic to obtain the experimental result of the experiment includes performing statistical processing on at least one of the traffic splitting result of the first requested traffic, the general indicators and private indicators of the first requested traffic to obtain the experimental result of the experiment. Thus, the experimental platform can perform statistical processing on the general indicators and / or private indicators of the first requested traffic, which helps to improve the comprehensiveness and accuracy of the experimental result.

[0252] Optionally, the method further includes displaying the statistical information of the first requested traffic for the user to understand the statistical information of the requested traffic.

[0253] The experimental traffic splitting method proposed by the present disclosure adds the identifier of the experiment and the scenario information of the experiment to the configuration information of the experiment. Among them, the experiment includes multiple candidate experiment versions, and the identifier of the experiment and the scenario information of the experiment are used to determine the target experiment version to be tested for the first request traffic from the multiple candidate experiment versions, and the configuration information of the experiment is sent to the service end according to the transmission mode of the pass-through parameters. Thereby, the identifier of the experiment and the scenario information of the experiment can be added to the configuration information of the experiment, which helps to improve the uniformity and orthogonality of experimental traffic splitting, and further improves the accuracy of experimental results. In addition, the configuration information of the experiment can be transmitted according to the transmission mode of the pass-through parameters, which can ensure the integrity and originality of the configuration information of the experiment.

[0254] In addition, by managing the configuration information of the experiment through the experimental platform, it is only necessary to develop corresponding interfaces on the service end to receive the configuration information of the experiment sent by the experimental platform.

[0255] In addition, when the configuration information of the experiment is dynamically adjusted, the service end only needs to continue to receive the adjusted configuration information of the experiment through the interface, without adjusting the service end code, which reduces the development cost and development time of the service end, and further reduces the experimental cost and experimental time.

[0256] Based on any of the above embodiments, as Figure 9 shown, the service end includes the SDK of the experimental platform, the user function module and the information processing module, the experimental platform includes the traffic splitting module and the information statistics module, and the information statistics module includes the general metric statistics unit, the private metric statistics unit and the information display unit.

[0257] Users can access the service end. Correspondingly, the user function module is used to generate request traffic in response to the user's access operation, and send the traffic splitting request of the request traffic to the SDK of the experimental platform deployed on the service end.

[0258] The SDK of the experimental platform deployed on the service end is used to obtain the configuration information of the experiment from the experimental platform, for example, send an information query request to the traffic splitting module and receive the configuration information of the experiment sent by the traffic splitting module.

[0259] The SDK of the experimental platform deployed on the service end is also used to determine multiple candidate experiment versions included in the experiment based on the configuration information of the experiment, extract the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment, and determine the target experiment version from the multiple candidate experiment versions based on the identifier of the request traffic, the identifier of the experiment and the scenario information of the experiment, and allocate the request traffic to the traffic group corresponding to the target experiment version.

[0260] The SDK of the experimental platform deployed on the service side is also used to generate the shunt result of the request traffic based on the target experimental version, store the shunt result of the request traffic in the local storage space, and feedback the shunt result of the request traffic to the user function module.

[0261] The user function module is used to conduct experiments based on the shunt result of the request traffic.

[0262] The information processing module is used to obtain the behavior information of the request traffic, obtain at least one of the general metrics and private metrics of the request traffic based on the behavior information of the request traffic as the buried point information of the request traffic, associate the buried point information of the request traffic with the shunt result of the request traffic to obtain the reported information of the request traffic, and send the reported information of the request traffic to the SDK of the experimental platform deployed on the service side.

[0263] The SDK of the experimental platform deployed on the service side is also used to send the reported information of the request traffic to the information statistics module.

[0264] The general metric statistical unit is used to perform statistical processing on the general metrics of the request traffic to obtain the statistical information of the general metrics of the request traffic.

[0265] The private metric statistical unit is used to perform statistical processing on the private metrics of the request traffic to obtain the statistical information of the private metrics of the request traffic.

[0266] The information display unit is used to display the reported information and / or statistical information of the request traffic.

[0267] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved are all in compliance with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0268] According to an embodiment of the present disclosure, the present disclosure also provides an experimental shunt device for implementing the above experimental shunt method.

[0269] Figure 10 It is a block diagram of an experimental shunt device according to an embodiment of the present disclosure.

[0270] As Figure 10 shown, the experimental shunt device 1000 includes: an acquisition module 1001, an extraction module 1002, a determination module 1003, and a storage module 1004.

[0271] The acquisition module 1001 is used to, in response to the first request traffic satisfying the shunt condition of the experiment, acquire the information of the first request traffic and the configuration information of the experiment, where the experiment includes multiple candidate experimental versions;

[0272] An extraction module 1002, configured to extract the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment;

[0273] A determination module 1003, configured to determine a target experiment version from multiple candidate experiment versions based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment;

[0274] A storage module 1004, configured to allocate the first request traffic to a traffic group corresponding to the target experiment version, and store the splitting result of the first request traffic in a local storage space.

[0275] In an embodiment of the present disclosure, the determination module 1003 is further configured to: determine a target splitting strategy of the experiment based on the scenario information of the experiment; process the information of the first request traffic and the identifier of the experiment according to the target splitting strategy to determine the target experiment version from multiple candidate experiment versions.

[0276] In an embodiment of the present disclosure, the determination module 1003 is further configured to: obtain a mapping relationship between candidate scenario information and candidate splitting strategies; obtain a similarity between the candidate scenario information and the scenario information of the experiment, and use the candidate scenario information with the largest similarity as the target scenario information; obtain a candidate splitting strategy that has a mapping relationship with the target scenario information as the target splitting strategy.

[0277] In an embodiment of the present disclosure, the determination module 1003 is further configured to: determine a scenario category of the experiment based on the scenario information of the experiment; in response to the scenario category of the experiment being the first category, determine that the traffic allocation ratio of the experimental group of the experiment is in a first interval; in response to the scenario category of the experiment being the second category, determine that the traffic allocation ratio of the experimental group of the experiment is in a second interval; in response to the scenario category of the experiment being the third category, determine that the traffic allocation ratio of the experimental group of the experiment is in a third interval; where

[0278] The upper limit value of the first interval is less than or equal to the lower limit value of the second interval, and the upper limit value of the second interval is less than or equal to the lower limit value of the third interval.

[0279] In one embodiment of the present disclosure, the determining module 1003 is further configured to: determine the scenario category of the experiment based on the scenario information of the experiment; determine the weight of each of the N attributes of the candidate shunting function based on the scenario category of the experiment, where N is an integer greater than or equal to 1; obtain the score of the candidate shunting function based on the N attributes of the candidate shunting function and the weights of the N attributes; and determine the target shunting function of the experiment from multiple candidate shunting functions based on the score.

[0280] In one embodiment of the present disclosure, the N attributes include a security level and a computing speed. When the scenario category of the experiment is the first category, the weight of the security level is higher than the weight of the computing speed.

[0281] When the scenario category of the experiment is the second category or the third category, the weight of the security level is lower than the weight of the computing speed.

[0282] In one embodiment of the present disclosure, the determining module 1003 is further configured to: obtain the target shunting strategy of the experiment; process the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment according to the target shunting strategy, so as to determine the target experiment version from multiple candidate experiment versions.

[0283] In one embodiment of the present disclosure, the determining module 1003 is further configured to: obtain the random number corresponding to the experiment; determine the target experiment version from multiple candidate experiment versions based on the information of the first request traffic, the identifier of the experiment, the scenario information of the experiment, and the random number corresponding to the experiment.

[0284] In one embodiment of the present disclosure, after the first request traffic is allocated to the traffic group corresponding to the target experiment version, the determining module 1003 is further configured to: perform semantic analysis on the configuration information of the experiment to obtain the object category of the experiment; determine the target data logging strategy of the first request traffic based on the object category of the experiment; and obtain the data logging information of the first request traffic according to the target data logging strategy.

[0285] In one embodiment of the present disclosure, the determining module 1003 is further configured to: obtain the mapping relationship between the candidate object categories and the candidate data logging strategies; and obtain the candidate data logging strategy that has a mapping relationship with the object category of the experiment as the target data logging strategy.

[0286] In one embodiment of the present disclosure, the determining module 1003 is further configured to: in response to the object category of the experiment being text, determine that the data logging information category of the first request traffic includes M categories.

[0287] In response to the object category of the experiment being an image, it is determined that the buried point information category of the first request traffic includes P categories;

[0288] In response to the object category of the experiment being voice, it is determined that the buried point information category of the first request traffic includes Q categories; where,

[0289] Both M, P, and Q are integers greater than or equal to 1;

[0290] Both the P categories and the Q categories at least include the M categories; or,

[0291] Both P and Q are greater than M, and there is partial overlap between the P categories and the M categories, and there is partial overlap between the Q categories and the M categories.

[0292] In an embodiment of the present disclosure, the buried point information category of the first request traffic includes at least one of general metrics and private metrics.

[0293] In an embodiment of the present disclosure, after storing the splitting result of the first request traffic in the local storage space, the determining module 1003 is further configured to: in response to the second request traffic meeting the splitting condition of the experiment; when the second request traffic meets at least one of the conditions that the identifier of the second request traffic is consistent with the identifier of the first request traffic and the second request traffic and the first request traffic originate from the same device, use the splitting result of the first request traffic in the local storage space as the splitting result of the second request traffic.

[0294] In an embodiment of the present disclosure, before using the splitting result of the first request traffic in the local storage space as the splitting result of the second request traffic, the determining module 1003 is further configured to: determine that the configuration information of the experiment has not changed.

[0295] In an embodiment of the present disclosure, when the second request traffic meets the at least one condition, the determining module 1003 is further configured to: in response to the configuration information of the experiment changing, obtain the splitting result of the second request traffic; replace the splitting result of the first request traffic in the local storage space with the splitting result of the second request traffic.

[0296] In an embodiment of the present disclosure, the determining module 1003 is further configured to: in response to the version number of the configuration information of the experiment changing, determine that the configuration information of the experiment has changed; in response to the version number of the configuration information of the experiment not changing, determine that the configuration information of the experiment has not changed.

[0297] In one embodiment of the present disclosure, the obtaining module 1001 is further configured to: receive the configuration information of the experiment sent by the experimental platform, where the configuration information of the experiment is transmitted according to the transmission mode of the pass-through parameters.

[0298] In one embodiment of the present disclosure, after allocating the first request traffic to the traffic group corresponding to the target experiment version, the storage module 1004 is further configured to: obtain the buried point information of the first request traffic; associate the buried point information of the first request traffic with the splitting result of the first request traffic to obtain the reporting information of the first request traffic; and send the reporting information of the first request traffic to the experimental platform.

[0299] In one embodiment of the present disclosure, after the first request traffic meets the splitting condition of the experiment, the obtaining module 1001 is further configured to: send a first indication information to the experimental platform, where the first indication information is used to indicate that the first request traffic meets the splitting condition of the experiment.

[0300] The experimental splitting device proposed by the present disclosure, in response to the first request traffic meeting the splitting condition of the experiment, obtains the information of the first request traffic and the configuration information of the experiment, where the experiment includes multiple candidate experiment versions, extracts the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment, determines the target experiment version from the multiple candidate experiment versions based on the information of the first request traffic, the identifier of the experiment and the scenario information of the experiment, allocates the first request traffic to the traffic group corresponding to the target experiment version, and stores the splitting result of the first request traffic in the local storage space. Thereby, it is possible to comprehensively consider the information of the request traffic, the identifier of the experiment and the scenario information of the experiment for experimental splitting, so that multiple experiments can be split according to their respective experiment identifiers and respective scenario information, which can ensure that the splitting results of the same request traffic in different experiments are different, helps to improve the uniformity and orthogonality of experimental splitting, and further improves the accuracy of experimental results, and is applicable to experimental splitting scenarios in fields such as finance, law, vehicles, smart homes, and entertainment.

[0301] In addition, storing the splitting result of the first request traffic in the local storage space can improve the efficiency of experimental splitting, quickly respond to user requests, optimize the user experience, and also reduce the computing resources occupied by experimental splitting on the service side, thereby helping to improve the response speed and concurrent processing ability of the service side.

[0302] According to an embodiment of the present disclosure, the present disclosure also provides another experimental splitting device for implementing the above experimental splitting method.

[0303] Figure 11 It is a block diagram of an experimental splitting device according to another embodiment of the present disclosure.

[0304] As Figure 11 shown, the experimental traffic splitting device 1100 includes: an adding module 1101 and a sending module 1102.

[0305] The adding module 1101 is used to add the identifier of the experiment and the scenario information of the experiment to the configuration information of the experiment. Among them, the experiment includes multiple candidate experiment versions, and the identifier of the experiment and the scenario information of the experiment are used to determine the target experiment version to be tested for the first request traffic from multiple candidate experiment versions;

[0306] The sending module 1102 is used to send the configuration information of the experiment to the service end according to the transmission method of the pass-through parameters.

[0307] In an embodiment of the present disclosure, the sending module 1102 is further used to: send the version number of the configuration information of the experiment to the service end.

[0308] In an embodiment of the present disclosure, the device 1100 further includes: a processing module. After sending the configuration information of the experiment to the service end, the processing module is used to: receive the reporting information of the first request traffic sent by the service end; and perform statistical processing on the reporting information of the first request traffic to obtain the experimental result of the experiment when receiving the first indication information sent by the service end, where the first indication information is used to indicate that the first request traffic meets the traffic splitting conditions of the experiment.

[0309] The experimental traffic splitting device proposed by the present disclosure adds the identifier of the experiment and the scenario information of the experiment to the configuration information of the experiment. Among them, the experiment includes multiple candidate experiment versions, and the identifier of the experiment and the scenario information of the experiment are used to determine the target experiment version to be tested for the first request traffic from multiple candidate experiment versions, and send the configuration information of the experiment to the service end according to the transmission method of the pass-through parameters. Thus, the identifier of the experiment and the scenario information of the experiment can be added to the configuration information of the experiment, which helps to improve the uniformity and orthogonality of experimental traffic splitting, and further improves the accuracy of experimental results. In addition, the configuration information of the experiment can be transmitted according to the transmission method of the pass-through parameters, which can ensure the integrity and originality of the configuration information of the experiment.

[0310] In addition, by managing the configuration information of the experiment through the experimental platform, only corresponding interfaces need to be developed on the service end to receive the configuration information of the experiment sent by the experimental platform.

[0311] In addition, when the configuration information of the experiment is dynamically adjusted, the service end only needs to continue to receive the adjusted configuration information of the experiment through the interface, without adjusting the service end code, reducing the development cost and development time of the service end, and further reducing the experimental cost and experimental time.

[0312] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0313] Figure 12 FIG. shows a schematic block diagram of an exemplary electronic device 1200 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0314] As Figure 12 shown, the device 1200 includes a computing unit 1201 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the device 1200 can also be stored. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0315] A plurality of components in the device 1200 are connected to the I / O interface 1205, including: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1206, such as various types of displays, speakers, etc.; a storage unit 1208, such as a magnetic disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the device 1200 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0316] The computing unit 1201 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 executes the various methods and processes described above, such as the experiment shunting method. For example, in some embodiments, the experiment shunting method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the experiment shunting method described above can be executed. Alternatively, in other embodiments, the computing unit 1201 can be configured to execute the experiment shunting method by any other suitable means (e.g., by means of firmware).

[0317] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), system-on-a-chip systems (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general programmable processor, that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0318] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0319] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0320] To present an interaction with a user account, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user account; and a keyboard and a pointing device (e.g., a mouse or a trackball), by which the user account can provide input to the computer. Other kinds of devices can also be used to present an interaction with the user account; for example, feedback presented to the user account can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input received from the user account can be in any form (including acoustic input, speech input, or tactile input).

[0321] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user account computer having a graphical user account interface or a web browser through which the user account can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of a communication network include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0322] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0323] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the steps of the experimental shunt method described in the above embodiments of the present disclosure are implemented.

[0324] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and no limitations are imposed herein.

[0325] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. An experimental traffic splitting method, comprising: In response to the first request traffic meeting the traffic splitting conditions of the experiment, obtaining the information of the first request traffic and the configuration information of the experiment, where the experiment includes multiple candidate experiment versions; Extracting the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment; Based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment, determining a target experiment version from multiple candidate experiment versions; Allocating the first request traffic to the traffic group corresponding to the target experiment version, and storing the traffic splitting result of the first request traffic in the local storage space.

2. The method according to claim 1, wherein, The determining a target experiment version from multiple candidate experiment versions based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment includes: Based on the scenario information of the experiment, determining the target traffic splitting strategy of the experiment; According to the target traffic splitting strategy, processing the information of the first request traffic and the identifier of the experiment to determine the target experiment version from multiple candidate experiment versions.

3. The method according to claim 2, wherein The determining the target traffic splitting strategy of the experiment based on the scenario information of the experiment includes: Obtaining the mapping relationship between candidate scenario information and candidate traffic splitting strategies; Obtaining the similarity between the candidate scenario information and the scenario information of the experiment, and taking the candidate scenario information with the maximum similarity as the target scenario information; Obtaining the candidate traffic splitting strategy that has a mapping relationship with the target scenario information as the target traffic splitting strategy.

4. The method according to claim 2, wherein The determining the target traffic splitting strategy of the experiment based on the scenario information of the experiment includes: Determining the scenario category of the experiment based on the scenario information of the experiment; In response to the scenario category of the experiment being the first category, determining that the traffic allocation ratio of the experimental group of the experiment is in the first interval; In response to the scenario category of the experiment being the second category, determining that the traffic allocation ratio of the experimental group of the experiment is in the second interval; In response to the scenario category of the experiment being the third category, determining that the traffic allocation ratio of the experimental group of the experiment is in the third interval; where The upper limit value of the first interval is less than or equal to the lower limit value of the second interval, and the upper limit value of the second interval is less than or equal to the lower limit value of the third interval.

5. The method according to claim 2, wherein, The determining the target traffic splitting strategy of the experiment based on the scenario information of the experiment includes: Determining the scenario category of the experiment based on the scenario information of the experiment; Based on the scenario category of the experiment, determining the weights of the N attributes of the candidate traffic splitting function respectively, where N is an integer greater than or equal to 1; Based on the N attributes of the candidate traffic splitting function and the weights of the N attributes respectively, obtaining the score of the candidate traffic splitting function; Based on the score, determining the target traffic splitting function of the experiment from multiple candidate traffic splitting functions.

6. The method according to claim 5, wherein The N attributes include the security level and the computing speed. In the case where the scenario category of the experiment is the first category, the weight of the security level is higher than the weight of the computing speed; When the scenario category of the experiment is the second category or the third category, the weight of the security level is lower than the weight of the calculation speed.

7. The method according to claim 1, wherein Determining a target experiment version from multiple candidate experiment versions based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment includes: Obtaining the target traffic splitting strategy of the experiment; Processing the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment according to the target traffic splitting strategy to determine the target experiment version from multiple candidate experiment versions.

8. The method according to claim 1, wherein, Determining a target experiment version from multiple candidate experiment versions based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment includes: Obtaining the random number corresponding to the experiment; Determining the target experiment version from multiple candidate experiment versions based on the information of the first request traffic, the identifier of the experiment, the scenario information of the experiment, and the random number corresponding to the experiment.

9. The method according to claim 1, wherein, After allocating the first request traffic to the traffic group corresponding to the target experiment version, it further includes: Performing semantic analysis on the configuration information of the experiment to obtain the object category of the experiment; Determining the target data tracking strategy of the first request traffic based on the object category of the experiment; Obtaining the data tracking information of the first request traffic according to the target data tracking strategy.

10. The method according to claim 9, wherein, Determining the target data tracking strategy of the first request traffic based on the object category of the experiment includes: Obtaining the mapping relationship between the candidate object categories and the candidate data tracking strategies; Obtaining the candidate data tracking strategy that has a mapping relationship with the object category of the experiment as the target data tracking strategy.

11. The method according to claim 9, wherein Determining the target data tracking strategy of the first request traffic based on the object category of the experiment includes: Responding to the object category of the experiment being text, determining that the data tracking information categories of the first request traffic include M categories; Responding to the object category of the experiment being an image, determining that the data tracking information categories of the first request traffic include P categories; Responding to the object category of the experiment being voice, determining that the data tracking information categories of the first request traffic include Q categories; where M, P, and Q are all integers greater than or equal to 1; The P categories and the Q categories both include at least the M categories; or, P and Q are both greater than M, and there is partial overlap between the P categories and the M categories, and there is partial overlap between the Q categories and the M categories.

12. The method according to claim 9, wherein The data tracking information categories of the first request traffic include at least one of general metrics and private metrics.

13. The method according to any one of claims 1 to 12, wherein, After storing the traffic splitting result of the first request traffic in the local storage space, it further includes: Responding to the second request traffic satisfying the traffic splitting condition of the experiment; When the second request traffic satisfies at least one of the condition that the identifier of the second request traffic is consistent with the identifier of the first request traffic and the second request traffic and the first request traffic come from the same device, using the traffic splitting result of the first request traffic in the local storage space as the traffic splitting result of the second request traffic.

14. The method according to claim 13, wherein Before using the splitting result of the first request traffic in the local storage space as the splitting result of the second request traffic, it further includes: Determining that the configuration information of the experiment has not changed.

15. The method according to claim 13, wherein, When the second request traffic meets the at least one condition, the method further includes: In response to a change in the configuration information of the experiment, obtaining the splitting result of the second request traffic; Replacing the splitting result of the first request traffic in the local storage space with the splitting result of the second request traffic.

16. The method according to claim 14, wherein The method further includes: In response to a change in the version number of the configuration information of the experiment, determining that the configuration information of the experiment has changed; In response to the version number of the configuration information of the experiment not changing, determining that the configuration information of the experiment has not changed.

17. The method according to any one of claims 1-12, wherein, Obtaining the configuration information of the experiment includes: Receiving the configuration information of the experiment sent by the experiment platform, where the configuration information of the experiment is transmitted according to the transmission method of the pass-through parameters.

18. The method according to any one of claims 1 to 12, wherein After allocating the first request traffic to the traffic group corresponding to the target experiment version, it further includes: Obtaining the buried point information of the first request traffic; Associating the buried point information of the first request traffic with the splitting result of the first request traffic to obtain the reporting information of the first request traffic; Sending the reporting information of the first request traffic to the experiment platform.

19. The method according to any one of claims 1 - 12, wherein After responding to the first request traffic meeting the splitting conditions of the experiment, it further includes: Sending the first indication information to the experiment platform, where the first indication information is used to indicate that the first request traffic meets the splitting conditions of the experiment.

20. An experiment splitting method includes: Adding the identifier of the experiment and the scenario information of the experiment to the configuration information of the experiment, where the experiment includes multiple candidate experiment versions, and the identifier of the experiment and the scenario information of the experiment are used to determine the target experiment version to be tested for the first request traffic from the multiple candidate experiment versions; Sending the configuration information of the experiment to the service end according to the transmission method of the pass-through parameters.

21. The method according to claim 20, wherein, The method further includes: Sending the version number of the configuration information of the experiment to the service end.

22. The method according to claim 20, wherein, After sending the configuration information of the experiment to the service end, it further includes: Receiving the reporting information of the first request traffic sent by the service end; When receiving the first indication information sent by the service end, performing statistical processing on the reporting information of the first request traffic to obtain the experimental result of the experiment, where the first indication information is used to indicate that the first request traffic meets the splitting conditions of the experiment.

23. An experiment splitting device includes: An acquisition module, configured to obtain the information of the first request traffic and the configuration information of the experiment in response to the first request traffic meeting the splitting conditions of the experiment, where the experiment includes multiple candidate experiment versions; An extraction module, configured to extract the identifier of the experiment and the scenario information of the experiment from the configuration information of the experiment; A determination module, configured to determine a target experiment version from multiple candidate experiment versions based on the information of the first request traffic, the identifier of the experiment, and the scenario information of the experiment; A storage module, configured to allocate the first request traffic to the traffic group corresponding to the target experiment version, and store the traffic splitting result of the first request traffic in the local storage space.

24. An experiment traffic splitting device, comprising: An addition module, configured to add the identifier of the experiment and the scenario information of the experiment to the configuration information of the experiment, where the experiment includes multiple candidate experiment versions, and the identifier of the experiment and the scenario information of the experiment are used to determine the target experiment version to be tested by the first request traffic from multiple candidate experiment versions; A sending module, configured to send the configuration information of the experiment to the service end according to the transmission method of the pass-through parameters.

25. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-22.

26. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-22.

27. A computer program product, comprising a computer program, where the computer program implements the method according to any one of claims 1-22 when executed by a processor.

Images