AB test gateway control method and system based on adaptive shunting, and storage medium

Abstract

The invention provides an AB test gateway control method and system based on adaptive shunting, and a storage medium, and the method comprises the steps: firstly obtaining a flow group through a preset weight grouping logic according to a user request, and guaranteeing the uniformity of flow distribution and the consistency of users; secondly, counting the triggering times of the conversion event according to a preset segmentation window, and dynamically calculating to obtain a conversion rate sequence; then, when the conversion rate deviation exceeds a threshold value, dynamically adjusting the grouping weight through a preset relational expression to realize self-adaptive progressive optimization; and finally, verifying the change trend of the conversion rate after the weight is adjusted, calculating the confidence coefficient by adopting statistical test, and automatically triggering a gray release process after the confidence coefficient reaches the standard. According to the method, the AB test is fused into a closed-loop process of adaptive statistical adjustment, the problem of uneven flow distribution is solved, and the decision response efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of gateway control, and more specifically, to a method, system, and storage medium for A / B testing gateway control based on adaptive traffic splitting. Background Technology

[0002] In the context of rapid iteration of internet products, A / B testing has become a core tool for verifying the effectiveness of features. However, traditional A / B testing relies on simple random algorithms, which can lead to grouping bias, especially when the user base is small, causing a sharp drop in the reliability of test conclusions. Furthermore, data collection and manual analysis can take hours or even days, missing the optimal release window. In addition, test logic needs to be embedded in the business code, increasing maintenance costs.

[0003] For example, patent CN115145801A discloses an A / B testing traffic allocation method, apparatus, device, and storage medium. The A / B testing traffic allocation method includes: acquiring feature information of pages to be deployed, wherein the pages to be deployed include at least two; inputting the feature information into a preset test prediction model, and performing conversion prediction processing on the feature information based on the test prediction model to obtain the predicted conversion performance value of the pages to be deployed in the next week, wherein the test prediction model is trained based on feature training samples, conversion performance labels of feature training samples, and time period labels of feature training samples; and determining the traffic allocation of the pages to be deployed based on the predicted conversion performance value. This scheme incorporates the time period labels of feature training samples during the training of the test prediction model, that is, it considers the time dimension in the process of predicting the conversion performance value of the pages to be deployed in the next week, thereby improving the accuracy of model prediction.

[0004] In addition, existing technologies have attempted to use cookie-based traffic distribution schemes, which improve the uniformity of distribution but do not solve the problem of real-time decision-making.

[0005] Therefore, there is an urgent need for a gateway-layer closed-loop adaptive traffic splitting technology for A / B testing, which can solve the problems of uneven traffic distribution and low decision-making response efficiency in A / B testing while controlling maintenance costs. Summary of the Invention

[0006] In view of the above problems, the purpose of this invention is to provide an A / B test gateway control method, system, and storage medium based on adaptive traffic splitting. This method achieves accurate traffic grouping by combining request ID, hash algorithm, and normalization algorithm; compresses analysis time through real-time statistics and comparative verification logic using a sliding window; reduces the cost of business code modification by adopting a non-intrusive event capture mechanism; establishes a deviation rate-weight dynamic mapping model to automatically optimize traffic allocation strategies; and provides accurate, automatic, and lightweight A / B testing technology for Internet products.

[0007] The first aspect of this invention provides an A / B test gateway control method based on adaptive traffic splitting, the method comprising: Receive and parse the user request to obtain the request ID; Based on the analysis of the request ID and the preset AB test ID pool, traffic grouping information is obtained through hash processing and combined with preset weight grouping logic; Based on the traffic grouping information, capture preset conversion events; Based on a preset segmentation window, the conversion rate is calculated according to the conversion event; Based on the test ID, query the conversion rate sequence and calculate the conversion deviation rate; Based on the analysis of the conversion deviation rate, the weight parameters of the weight grouping logic are dynamically adjusted; Calculate the conversion rate deviation before and after the weight parameters are adjusted, and combine it with the adjusted weight parameters to obtain the confidence information; Based on the analysis of confidence information and the generation of a decision report using the test ID, a test version is released according to traffic grouping information.

[0008] Furthermore, based on the analysis of the request ID and the preset A / B test ID pool, and combined with the preset weight grouping logic, traffic grouping information is obtained, specifically: Parse the user request to extract the request ID and test ID; Based on a preset hash algorithm, the request ID and test ID are merged and hashed to obtain a hash value; Take a consecutive byte substring from the hash value, calculate the ratio of the value of the consecutive byte substring to the maximum feasible value of the corresponding consecutive byte substring, and give the mapping value; Divide the grouping intervals according to the weight parameters of the weight grouping logic; Based on the grouping range to which the mapped value belongs, the user request is associated with the traffic group, and traffic group information is given, wherein the traffic group includes at least group A and group B.

[0009] Furthermore, the request ID is extracted, which further includes: Based on the request ID, obtain the device type, user attributes, and geographic region information; Based on the timestamp, device type, population attributes, and geographical area information, label dimensions are defined, the encoding of each label dimension is determined, and the request ID is processed to form a multidimensional structured label.

[0010] Furthermore, the step of capturing preset conversion events based on the traffic grouping information specifically includes: The traffic grouping information is matched with a predefined set of conversion event rules, wherein the set of conversion event rules includes feature interfaces, feature fields and feature parameters; If a conversion event request from the feature interface is received, the field parameters are obtained by parsing the feature fields. Determine whether the field parameter is the same as the feature parameter; If so, it means that a conversion event has been captured.

[0011] Furthermore, based on a preset segmentation window, the conversion rate is calculated according to the conversion event, specifically as follows: Based on the segmentation window, the total number of requests and the number of events triggered by the traffic group information are counted, wherein the segmentation window is a preset time window or the structured label; The conversion rate is obtained by calculating the ratio of the number of event triggers to the total number of requests.

[0012] Furthermore, the step of dynamically adjusting the weight parameters of the weight grouping logic based on the conversion deviation rate specifically involves: Obtain the first dynamic adjustment relation; Based on the first dynamic adjustment relationship, the first adjustment weight is obtained according to the conversion deviation rate; The first weight threshold is obtained based on the current weight parameters of the weight grouping logic; Based on the analysis of the first adjustment weight and the first weight threshold, the adjustment weight parameters of the weight grouping logic are given.

[0013] Furthermore, the first dynamic adjustment relationship is specifically expressed as: ; Where, Δw A The adjustment magnitude of the weight parameters for group A, Δw B Here, K is the adjustment magnitude of the weight parameters for group B, and C is the step size parameter. A Let C be the conversion rate of group A. B The conversion rate for group B; Adjusting the weight parameters is specifically expressed as follows: ; Among them, w A,T w is the adjusted weighting parameter for group A. B,T For the adjusted weighting parameters of group B, w A,C Here are the current weight parameters for group A, where α is the upper limit of the weight ratio coefficient, and w max To adjust the upper limit of the weight, w B,C Here are the current weight parameters for group B, and min is the function to find the minimum value.

[0014] Furthermore, after obtaining the first dynamic adjustment relation, the specific details include: Generate a conversion rate sequence dataset based on the current conversion rate sequence and the historical conversion rate sequence; Feature processing is performed on the conversion rate sequence dataset to form conversion rate features, and traffic features and system features are obtained to give time-series features. Combined with static features, multi-feature information is formed. The multi-feature information includes time-series features and static features. The time-series features include conversion rate features, traffic features and system features, and the static features include user features and test features. The first dynamic adjustment relationship and multi-feature information are input into the preset neural network model, and the first dynamic adjustment relationship is updated based on the historical trend adjustment factor, multi-objective adjustment factor and correction factor.

[0015] Furthermore, the neural network model includes an input layer, a processing layer, a feature fusion layer, and an output layer. The processing layer includes a time series branch and a static feature branch. The time series branch includes at least two LSTM layers, and the static feature branch includes at least two fully connected layers.

[0016] Furthermore, the first dynamic adjustment relation is updated, specifically expressed as: ; Where, Δw A K(t) represents the first dynamic adjustment relation after the update at time frame t, where K(t) is the step size parameter for time frame t, and C... A (t) represents the conversion rate of group A at time frame t, and C B (t) represents the conversion rate of group B at time frame t, f(ΔC) hist (t) is the historical trend adjustment factor for time frame t, g(D(t), R(t)) is the multi-objective adjustment factor for time frame t, and h(ΔC) is the historical trend adjustment factor for time frame t. hist (t) is the correction factor for time frame t; The step size parameter K(t) of time frame t is specifically expressed as: ; Where K0 is the basic step size parameter, V(t) is the traffic fluctuation index of time frame t, N(t) is the proportion of new users in time frame t, and m and n are the feature weights of the traffic fluctuation index and the proportion of new users, respectively. Historical trend adjustment factor f(ΔC) for time frame t hist (t) is specifically represented as: ; Where T is the number of historical windows. For trend weights, C A (ti) represents the conversion rate of group A in time frame ti, and C B(ti) represents the conversion rate of group B in time frame ti; The multi-objective adjustment factor g(D(t), R(t)) for time frame t is specifically expressed as: ; Where D(t) is the system latency index for time frame t, and R(t) is the user retention rate index for time frame t. These are the feature weights for the system latency index and the user retention rate index, respectively. Correction factor h (ΔC) for time frame t hist (t) is specifically represented as: ; Where T is the number of historical windows. To adjust the weights, C A (ti) represents the conversion rate of group A in time frame ti, and C B (ti) represents the B-group conversion rate of time frame ti.

[0017] Furthermore, the conversion rate deviation before and after the weight parameter adjustment is combined with the adjusted weight parameter to obtain confidence information, specifically: Based on the conversion rate deviation, determine whether the conversion rate after adjusting the weight parameters shows a downward trend; If so, the confidence level information is set to a preset first confidence reference value; If not, then the first weight information is determined based on the conversion rate deviation; The confidence level information is calculated based on the first weight information and the weight parameters according to a preset weighting algorithm.

[0018] Furthermore, based on the first weight information and the weight parameters, confidence information is calculated according to a preset weighting algorithm, specifically as follows: ; Where CI is the confidence level, min is the minimum function, max is the maximum function, and w T For the adjusted weight parameters, C T To adjust the conversion rate, C C To adjust the conversion rate before adjustment, M min M is the lower confidence level. max This represents the upper limit of confidence.

[0019] A second aspect of the present invention provides an AB test gateway control system based on adaptive flow splitting, including an AB test gateway control method program based on adaptive flow splitting, wherein the AB test gateway control method program based on adaptive flow splitting, when executed by the processor, implements the following steps: Receive and parse the user request to obtain the request ID; Based on the analysis of the request ID and the preset AB test ID pool, traffic grouping information is obtained through hash processing and combined with preset weight grouping logic; Based on the traffic grouping information, capture preset conversion events; Based on a preset segmentation window, the conversion rate is calculated according to the conversion event; Based on the test ID, query the conversion rate sequence and calculate the conversion deviation rate; Based on the analysis of the conversion deviation rate, the weight parameters of the weight grouping logic are dynamically adjusted; Calculate the conversion rate deviation before and after the weight parameters are adjusted, and combine it with the adjusted weight parameters to obtain the confidence information; Based on the analysis of confidence information and the generation of a decision report using the test ID, a test version is released according to traffic grouping information.

[0020] A third aspect of the present invention provides a computer-readable storage medium comprising an A / B test gateway control method program based on adaptive flow splitting, wherein when the A / B test gateway control method program based on adaptive flow splitting is executed by a processor, it implements the steps of the A / B test gateway control method based on adaptive flow splitting as described in any of the preceding claims.

[0021] This invention provides an A / B testing gateway control method, system, and storage medium based on adaptive traffic splitting. First, traffic is grouped according to user requests using a preset weighted grouping logic to ensure uniform traffic distribution and user consistency. Second, the conversion rate sequence is dynamically calculated based on the number of conversion events triggered by a preset segmentation window. Then, when the conversion rate deviation exceeds a threshold, the grouping weights are dynamically adjusted using a preset relational formula to achieve adaptive incremental optimization. Finally, the conversion rate trend is verified after weight adjustment, and a confidence level is calculated using statistical tests. Once the confidence level meets the standard, a canary release process is automatically triggered. This invention integrates A / B testing into a closed-loop process of adaptive statistical adjustment, solving the problem of uneven traffic distribution and improving decision-making response efficiency. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope.

[0023] Figure 1 A flowchart of an A / B test gateway control method based on adaptive traffic splitting according to the present invention is shown; Figure 2 A flowchart illustrating the allocation of traffic grouping information provided in an embodiment of the present invention is shown; Figure 3 The following is a flowchart illustrating the execution of capturing conversion events according to an embodiment of the present invention; Figure 4 This invention provides a flowchart illustrating the weight parameters of a weight grouping logic that dynamically adjusts the weights based on the conversion deviation rate, according to an embodiment of the invention. Figure 5 An architecture diagram of the neural network model provided in an embodiment of the present invention is shown; Figure 6 A block diagram of an AB test gateway control system based on adaptive flow splitting according to the present invention is shown. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Unless otherwise defined, all terms (including technical and scientific terms) used in embodiments of this invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in a common dictionary shall be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as being interpreted in an idealized or highly formalized sense, unless expressly defined in this embodiment of the invention.

[0026] The terms "first," "second," and similar words used in the embodiments of this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "an," "a," or "the" do not indicate a quantity limitation, but rather indicate the presence of at least one. Similarly, terms such as "including" or "comprising" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The steps preceding or following the steps in the method of the embodiments of this invention are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0027] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0028] Figure 1 A flowchart of an AB test gateway control method based on adaptive traffic splitting according to the present invention is shown.

[0029] like Figure 1 As shown, the first aspect of this invention discloses an A / B test gateway control method based on adaptive traffic splitting, the method comprising: S102, Receive and parse the user request to obtain the request ID; S104. Based on the analysis of the request ID and the preset AB test ID pool, traffic grouping information is obtained through hash processing and combined with preset weight grouping logic; that is, if the request ID falls into the preset AB test ID pool, traffic grouping information is obtained according to the user request based on the preset weight grouping logic. S106, based on the traffic grouping information, capture a preset conversion event; S108, Based on a preset segmentation window, the conversion rate is calculated according to the conversion event; S110, based on the test ID, query the conversion rate sequence and calculate the conversion deviation rate; S112, Based on the analysis of the conversion deviation rate, dynamically adjust the weight parameters of the weight grouping logic; that is, determine whether the conversion deviation rate exceeds a preset deviation rate threshold, and if so, dynamically adjust the weight parameters of the weight grouping logic according to the conversion deviation rate. S114, calculate the conversion rate deviation before and after the weight parameters are adjusted, and combine it with the adjusted weight parameters to obtain the confidence information; S116. Based on the analysis of confidence information and combined with the test ID, a decision report is generated, and a test version is released according to the traffic grouping information. That is, it is determined whether the confidence information exceeds the preset confidence threshold. If so, a decision report is generated according to the test ID, and a test version is released according to the traffic grouping information.

[0030] It should be noted that the AB test ID pool is the AB test whitelist. An AB test includes at least two traffic groupings: group A and group B. This embodiment provides an AB test gateway control flow based on adaptive traffic splitting. It receives user feedback on AB tests via network requests and selects the release version based on the feedback. First, it receives the user's request link from the gateway, parses and extracts the request ID from the request link, and checks whether the request ID is the same as a test ID in the preset AB test ID pool. If they are different, the request will not trigger an AB test; if they are the same, the corresponding AB test is performed based on the matched test ID. For ease of explanation, let's take the impact of button color on click-through rate as an example to explain AB testing: Group A is red buttons, and Group B is blue buttons. The button color for release is selected based on the user's click-through rate. After matching the test ID corresponding to the button color, the corresponding group is selected to display the button color according to the preset weight grouping logic based on the data in the user request. The initial grouping ratio is A:B = 1:1. Subsequently, the number of times users clicked the buttons is counted. Based on a preset segmented window, the total number of requests and clicks within the window are counted to calculate the click-through rate (CTR), which is the conversion rate of the conversion event. The segmented window includes, but is not limited to, time windows, geographic windows, and user attribute windows. Then, based on the segmented window, the CTRs of the red and blue buttons are retrieved separately to obtain a conversion rate sequence. This sequence is used to calculate the CTR deviation between the red and blue buttons, which is the conversion deviation rate. When the CTR deviation between the red and blue buttons exceeds a preset deviation rate threshold, it indicates that the button color has a certain influence on the CTR. In this case, the weight parameters of the weighted grouping logic are adjusted based on the CTR deviation. For example, if the CTR of the red button is higher than that of the blue button, the weight parameters are adjusted to increase the probability of entering the red button, i.e., increasing the group traffic of group A. Furthermore, increasing the sample size further improves the authenticity of the test results. The conversion rate deviation of Group A after increasing traffic to Group A is calculated, i.e., the click-through rate (CTR) change of the red button is calculated. Based on the CTR change and the weight parameters of the grouping logic, a preset weighted calculation is performed to obtain confidence information. The confidence information represents the effectiveness of the grouping test; the higher the confidence, the greater the impact of the corresponding group on the conversion event. For example, a higher confidence for Group A indicates that the red button can improve the CTR. When the confidence for Group A exceeds a preset confidence threshold, a decision report on whether Group A can be launched is generated, and Group A is launched according to a preset release process. In other words, the red button will be used as the launch button color. This embodiment is based on real-time data-driven weight adjustment, avoiding manual intervention; and based on statistical testing, it ensures the reliability of A / B testing and reduces the risk of misjudgment; furthermore, the conclusions of A / B testing directly generate decisions and are released, shortening the iteration cycle.

[0031] Figure 2A flowchart illustrating the allocation of traffic grouping information provided in an embodiment of the present invention is shown.

[0032] According to embodiments of the present invention, such as Figure 2 As shown, based on the analysis of the request ID and the preset AB test ID pool, and combined with the preset weight grouping logic, traffic grouping information is obtained, specifically: S202, parse the user request and extract the user ID and test ID; S204, based on a preset hash algorithm, merges and hashes the request ID and test ID to obtain a hash value; S206, take a continuous byte substring from the hash value, calculate the ratio of the value of the continuous byte substring to the maximum feasible value of the corresponding continuous byte substring, and give the mapping value; S208, Divide the grouping interval according to the weight parameters of the weight grouping logic; S210, based on the grouping interval to which the mapped value belongs, associate the user request with the traffic group and provide traffic group information, wherein the traffic group includes at least group A and group B.

[0033] The ratio of the numerical value of a consecutive byte substring to the maximum feasible value of the corresponding consecutive byte substring is specifically expressed as: ; Where x is the mapped value, v is the value of the consecutive byte substring, N is the base of each byte, and i is the length of the consecutive byte substring. The value of the consecutive byte substring is generally obtained through base conversion, N i -1 represents the maximum feasible value of the corresponding consecutive byte substring during base conversion. For example, the consecutive byte substring "0x5f4d" in hexadecimal occupies 2 bytes. The value v of the consecutive byte substring is obtained through decimal conversion, specifically 5 × 16. 3 +15×16 2 +4×16 1 +13×16 0 =24429, N i -1 represents the maximum feasible value corresponding to "0x5f4d", in which case N is 2. 8 (Each byte has 8 bits), i is 2, representing the length of the 2 bytes occupied by the consecutive byte substring "0x5f4d", N i -1 is 65535.

[0034] Based on the grouping range to which the mapped value belongs, the user request is associated with the traffic group, specifically as follows: ; Where Group is the grouping function, A is group A of the traffic groups, B is group B of the traffic groups, I is the indicator function, and w A,C w represents the current weight parameters for group A. B,C These are the current weight parameters for group B.

[0035] It should be noted that this embodiment provides a weighted grouping logic. In this embodiment, firstly, the user ID and the ID of the hit test are extracted according to the user request. Then, based on the user ID and the test ID, a hash value is obtained according to a preset hash algorithm; then, according to a preset normalization process, it is converted into a normalized value in the interval [0,1]. As one implementation, the user ID is "user_123" and the test ID is "exp_102". First, the user ID and the test ID are concatenated as strings to generate a composite string "user_123_exp_102"; then, the composite string is hashed according to a preset salted MD5 hash to obtain a 128-bit hash value "0x5f4dcc3b5aa765d61d8327deb882cf99"; one or more bytes in the hash value are extracted and mapped to the interval [0,1], such as the normalized mapping value of "0x5f4d" being "0.37". Then, based on the weight parameters of the weighted grouping logic, the grouping intervals are divided. As one implementation, when the weight parameter is 0.5, group A is divided into intervals [0, 0.5] and [0.5, 1.0]. Since the normalized value "0.37" falls within the interval [0, 0.5], the traffic grouping information is group A. This example ensures a uniform distribution of user groups through a hash algorithm, and the preset normalization design provides a basis for expanding groups, such as adding groups C and D.

[0036] According to an embodiment of the present invention, extracting the request ID further includes: Based on the request ID, obtain the device type, user attributes, and geographic region information; Based on the timestamp, device type, population attributes, and geographical area information, label dimensions are defined, the encoding of each label dimension is determined, and the request ID is processed to form a multidimensional structured label.

[0037] It should be noted that this embodiment also provides a user profiling mechanism. A user profile is created by binding a corresponding structured tag to the user ID in the user request. This structured tag includes at least time, device type, demographic attributes, and geographic region attributes. Device type can be PC, mobile phone, tablet, etc.; demographic attributes include gender, VIP level, etc. Data is aggregated by tag dimension, for example, to calculate conversion rates under different tags. This embodiment uses multi-dimensional tags to differentiate user segmentation effects and identify high-value groups; furthermore, it uses region and device tags to filter environmental interference data, improving the accuracy of data analysis.

[0038] Figure 3 The following is a flowchart illustrating the execution of capturing conversion events according to an embodiment of the present invention.

[0039] According to embodiments of the present invention, such as Figure 3 As shown, capturing preset conversion events based on the traffic grouping information specifically involves: S302, Match a predefined set of conversion event rules according to the traffic grouping information, wherein the set of conversion event rules includes feature interfaces, feature fields and feature parameters; S304, if a conversion event request from the feature interface is received, the field parameters are obtained by parsing the feature fields; S306, determine whether the field parameter is the same as the feature parameter; S308, if yes, indicates that a conversion event has been captured.

[0040] It should be noted that, as one implementation method, let's take A / B testing of button colors as an example. The data transmission link corresponding to the button is the feature interface. In the feature interface, the button press flag is the feature field, and the feature parameter is true. That is, if the button is pressed, a link with the feature field set to true will be generated through the feature interface as a conversion event request. When a link request is received, the parameter in the feature field of the request is parsed and compared with the feature parameter; if they are the same, it means that the button has been pressed, that is, a conversion event has been captured.

[0041] According to embodiments of the present invention, such as Figure 4 As shown, the step of dynamically adjusting the weight parameters of the weight grouping logic based on the conversion deviation rate specifically involves: Obtain the first dynamic adjustment relation; Based on the first dynamic adjustment relationship, the first adjustment weight is obtained according to the conversion deviation rate; The first weight threshold is obtained based on the current weight parameters of the weight grouping logic; Based on the analysis of the first adjustment weight and the first weight threshold, the adjustment weight parameters of the weight grouping logic are given.

[0042] That is, determine whether the first adjustment weight is greater than the first weight threshold; If so, the weight parameters of the weight grouping logic are adjusted according to the first weight threshold. If not, then adjust the weight parameters of the weight grouping logic according to the first adjustment weight.

[0043] The first dynamic adjustment relationship is specifically expressed as follows: ; Where, Δw A The adjustment magnitude of the weight parameters for group A, Δw B Here, K is the adjustment magnitude of the weight parameters for group B, and C is the step size parameter. A Let C be the conversion rate of group A. B The conversion rate for group B; Adjusting the weight parameters is specifically expressed as follows: ; Among them, w A,T w is the adjusted weighting parameter for group A. B,T For the adjusted weighting parameters of group B, w A,C Here are the current weight parameters for group A, where α is the upper limit of the weight ratio coefficient, and w max To adjust the upper limit of the weight, w B,C Here are the current weight parameters for group B, and min is the function to find the minimum value.

[0044] It should be noted that this embodiment provides a logic for dynamically adjusting weight parameters. In this embodiment, firstly, a first dynamic adjustment formula stored in memory is obtained, and the weight adjustment amount, i.e., the first adjustment weight, is calculated based on the conversion deviation rate. Then, based on the current weight parameters, an upper limit value for weight adjustment, i.e., a first weight threshold, is determined to avoid overshooting and excessive fluctuations in statistical data. When the weight adjustment amount is higher than the upper limit, the weight parameters of the weight grouping logic are adjusted according to the upper limit; when the weight adjustment amount is not higher than the upper limit, the weight parameters of the weight grouping logic are adjusted according to the weight adjustment amount. This embodiment controls the adjustment amplitude by adjusting the formula and the upper limit value to prevent traffic oscillations, and adopts an adaptive small-step, fast-run strategy to adjust the weight parameters step by step, thereby improving the stability of statistical data.

[0045] According to an embodiment of the present invention, the conversion rate deviation before and after the weight parameter adjustment is calculated, and the confidence information is obtained by combining the adjusted weight parameter, specifically as follows: Based on the conversion rate deviation, determine whether the conversion rate after adjusting the weight parameters shows a downward trend; If so, the confidence level information is set to a preset first confidence reference value; If not, then the first weight information is determined based on the conversion rate deviation; The confidence level information is calculated based on the first weight information and the weight parameters according to a preset weighting algorithm.

[0046] Specifically, the confidence level information is calculated based on the first weight information and the weight parameters using a preset weighting algorithm, as follows: ; Where CI is the confidence level, min is the minimum function, max is the maximum function, and w T For the adjusted weight parameters, C T To adjust the conversion rate, C C To adjust the conversion rate before adjustment, M min M is the lower confidence level. max This represents the upper limit of confidence.

[0047] It should be noted that this embodiment provides the logic for calculating the confidence level. As one implementation method, the change in the conversion rate of group A before and after adjusting the weight parameters is calculated. If the conversion rate shows a downward trend, it indicates that the weight parameter adjustment has a negative feedback effect on the conversion rate of group A, and the confidence level is set to a first confidence reference value, where 30% or similar values ​​represent lower confidence levels. If the conversion rate remains flat or shows an upward trend, it indicates that the weight parameter adjustment has a positive feedback effect on the conversion rate of group A. In this case, the first weight information is determined based on the conversion rate deviation, and used for weighted calculation with the weight ratio to obtain the confidence level information.

[0048] It is worth mentioning that, based on the preset segmentation window, the conversion rate is calculated according to the conversion event, specifically as follows: Based on the segmentation window, the total number of requests and the number of events triggered by the traffic group information are counted, wherein the segmentation window is a preset time window or the structured label; The conversion rate is obtained by calculating the ratio of the number of event triggers to the total number of requests.

[0049] It should be noted that in this embodiment, different segmentation windows are used based on the different A / B test items. The default segmentation window type is a time window, which divides the data stream according to a preset time interval to generate statistical data. The segmentation window type can also be a label window, for example, dividing the window according to the user's VIP level. When the cumulative request volume of a VIP level exceeds a preset value, the window is closed.

[0050] In addition, it includes a dynamic window adjustment mechanism; when the window request traffic rate growth rate exceeds a preset first growth rate threshold, the window time is automatically shortened; when the window request traffic rate growth rate is lower than a preset second growth rate threshold, the window time is automatically extended.

[0051] It's worth mentioning that this also includes gradually releasing test versions according to a pre-defined gray-scale phase strategy, specifically: Set up a gray-scale phase according to the preset release cycle; In each gray-scale phase, a test version is released according to a preset percentage step. Monitor the conversion rate change rate after each gray-scale release; If the rate of change of the conversion rate in any stage is less than the preset rate of change threshold, the release will be terminated and the A / B test will be restarted.

[0052] It should be noted that this embodiment provides a phased release strategy. As one implementation method, a 6-hour release cycle is used, with 10% of traffic directed to the new version in each cycle. Simultaneously, a monitoring dashboard is activated to track key metrics, such as the conversion rate change rate after each phase of the phased release. If the conversion rate change rate is less than a preset threshold, it indicates a significant drop in conversion rate after the new version release; in this case, the release is terminated, a rollback is immediately triggered, and A / B testing is repeated. This embodiment releases traffic in stages, ensuring that local failures do not affect the overall system; furthermore, the gradual release reduces disruptions to the user experience.

[0053] It is worth mentioning that after obtaining the first dynamic adjustment relation, it specifically includes: Generate a conversion rate sequence dataset based on the current conversion rate sequence and the historical conversion rate sequence; Feature processing is performed on the conversion rate sequence dataset to form conversion rate features, and traffic and system features are obtained to provide time-series features. Combined with static features, multi-feature information is formed. The multi-feature information includes time-series features, traffic features, and system features. Static features include user features and test features. Traffic features may include traffic fluctuation index, request burst coefficient, etc. System features may include system latency, etc. User features may include user groups, user attributes, etc. Test features may include test type, test stage, etc. The multi-feature information is concatenated according to time windows.

[0054] The first dynamic adjustment relationship and multi-feature information are input into the preset neural network model, and the first dynamic adjustment relationship is updated based on the historical trend adjustment factor, multi-objective adjustment factor and correction factor.

[0055] like Figure 5As shown, the neural network model includes an input layer, a processing layer, a feature fusion layer, and an output layer. The processing layer includes a time series branch and a static feature branch. The time series branch includes at least two LSTM layers, and the static feature branch includes at least two fully connected layers.

[0056] The time series branch primarily processes temporal features, extracting dependencies along the time dimension, while the static feature branch primarily processes static features, extracting their semantic information. The feature fusion layer fuses temporal and static features through a multi-head self-attention mechanism, highlighting features more relevant to the decision-making process in the current time frame.

[0057] It should be noted that this embodiment provides a dynamic optimization mechanism for neural networks. As one implementation method, the conversion rate sequences synthesized from the current conversion rates of groups A and B, along with historical sequences, are combined to form a conversion rate sequence dataset. Multi-feature information is input into a preset LSTM neural network model to learn working characteristics, such as conversion rate fluctuations during peak weekday hours and late-night fluctuations. The LSTM neural network model updates and replaces the original first dynamic adjustment formula, improving the efficiency and accuracy of adjusting the weight parameters.

[0058] The first dynamic adjustment relation is updated, specifically expressed as: ; Where, Δw A K(t) represents the first dynamic adjustment relation after the update at time frame t, where K(t) is the step size parameter for time frame t, and C... A (t) represents the conversion rate of group A at time frame t, and C B (t) represents the conversion rate of group B at time frame t, f(ΔC) hist (t) is the historical trend adjustment factor for time frame t, g(D(t), R(t)) is the multi-objective adjustment factor for time frame t, and h(ΔC) is the historical trend adjustment factor for time frame t. hist (t) is the correction factor for time frame t; The step size parameter K(t) of time frame t is specifically expressed as: ; Wherein, K0 is the basic step size parameter, V(t) is the traffic fluctuation index of time frame t, for example, it can be the current window traffic or the average traffic of the past 24 hours. The larger the value, the greater the traffic fluctuation. N(t) is the proportion of new users in time frame t, which is the number of new users in the current window / the total number of users. The larger the value, the more unstable the user behavior. m and n are the feature weights of the traffic fluctuation index and the proportion of new users, respectively. The specific values ​​of m and n can be obtained through offline training of the neural network model. The step size parameters, based on the traffic fluctuation index and the proportion of new users, can be dynamically adjusted to adapt to different testing scenarios (such as more cautious adjustments during peak hours and faster adjustments when the proportion of new users is high).

[0059] Historical trend adjustment factor f(ΔC) for time frame t hist (t) is obtained by taking the mean difference in conversion rates over the most recent T windows, specifically expressed as: ; Where T is the number of historical windows. For trend weighting, η >0 indicates that when the historical trend shows Group A continuously improving, the weight adjustment range is increased, C A (ti) represents the conversion rate of group A in time frame ti, and C B (ti) represents the conversion rate of group B in time frame ti; The historical trend adjustment factor is based on historical data to avoid misjudgment due to "single point data" (e.g., when the conversion rate of a certain window suddenly increases but the trend declines, the adjustment range will be reduced).

[0060] The multi-objective adjustment factor g(D(t), R(t)) for time frame t is used to balance conversion rate with system latency and retention rate, and is specifically expressed as follows: ; Where D(t) is the system latency index at time frame t, which is the current window system latency / latency threshold. The larger the value, the greater the system load. R(t) is the user retention rate index at time frame t, which is the current window retention rate / baseline retention rate. The smaller the value, the more severe the user churn. These are the feature weights for the system latency index and the user retention rate index, respectively. The specific values ​​can be obtained through offline training of the neural network model; Correction factor h (ΔC) for time frame t hist (t)), using historical trends to adjust the weights of Group B, specifically expressed as: ; Where T is the number of historical windows. To adjust the weights, This indicates that as group A continues to improve, the weight of group B decreases more significantly, and group C... A (ti) represents the conversion rate of group A in time frame ti, and C B (ti) represents the B-group conversion rate of time frame ti.

[0061] The impact of traffic fluctuations and abnormal data is mitigated by using multi-objective adjustment factors and B-group correction factors to prevent excessive weight adjustment (e.g., the adjustment range will automatically decrease during traffic peaks).

[0062] It is worth mentioning that the preset for the neural network model also includes: training the neural network model until it meets the threshold requirement of the multi-objective loss function, which is specifically expressed as: ; in, Let be the target weights, and let their sum be 1. Mean squared error (used for conversion rate improvement optimization). Mean absolute error (used as a constraint for system latency and retention rate). The conversion rates before and after the adjustment are respectively. The system latency before and after adjustment These are the user retention rates before and after the adjustment.

[0063] By using a multi-objective loss function to train and update the neural network model, we avoid the "short-sighted decision-making" caused by a single conversion rate objective. Through multi-objective optimization, we balance business objectives and system constraints, and improve the accuracy and rationality of weight adjustment.

[0064] Figure 6 A block diagram of an AB test gateway control system based on adaptive flow splitting according to the present invention is shown.

[0065] like Figure 6 As shown, the second aspect of the present invention discloses an AB test gateway control system 4 based on adaptive flow splitting, including a memory 41 and a processor 42. The memory includes an AB test gateway control method program based on adaptive flow splitting. When the processor executes the AB test gateway control method program based on adaptive flow splitting, it performs the following steps: Receive and parse the user request to obtain the request ID; Based on the analysis of the request ID and the preset AB test ID pool, traffic grouping information is obtained through hash processing and combined with preset weight grouping logic; Based on the traffic grouping information, capture preset conversion events; Based on a preset segmentation window, the conversion rate is calculated according to the conversion event; Based on the test ID, query the conversion rate sequence and calculate the conversion deviation rate; Based on the analysis of the conversion deviation rate, the weight parameters of the weight grouping logic are dynamically adjusted; Calculate the conversion rate deviation before and after the weight parameters are adjusted, and combine it with the adjusted weight parameters to obtain the confidence information; Based on the analysis of confidence information and the generation of a decision report using the test ID, a test version is released according to traffic grouping information.

[0066] It should be noted that A / B testing includes at least two groups of traffic grouping information, namely group A and group B. This embodiment provides an A / B testing gateway control flow based on adaptive traffic splitting. It receives user feedback on A / B testing operations via network requests and selects the release version based on the feedback. First, it receives the user's request link from the gateway, parses and extracts the request ID from the request link, and checks whether the request ID is the same as a test ID in a preset A / B test ID pool. If they are different, the request will not trigger an A / B test; if they are the same, the corresponding A / B test is performed based on the matched test ID. For ease of explanation, let's take the impact of button color on click-through rate as an example to explain A / B testing: Group A is red buttons, and Group B is blue buttons. The button color for release is selected based on the user's click-through rate. After matching the test ID corresponding to the button color, the corresponding group is selected to display the button color according to the preset weight grouping logic based on the data in the user request. The initial grouping ratio is A:B = 1:1. Subsequently, the number of times users clicked the buttons is counted. Based on a preset segmented window, the total number of requests and clicks within the window are counted to calculate the click-through rate (CTR), which is the conversion rate of the conversion event. The segmented window includes, but is not limited to, time windows, geographic windows, and user attribute windows. Then, based on the segmented window, the CTRs of the red and blue buttons are retrieved separately to obtain a conversion rate sequence. This sequence is used to calculate the CTR deviation between the red and blue buttons, which is the conversion deviation rate. When the CTR deviation between the red and blue buttons exceeds a preset deviation rate threshold, it indicates that the button color has a certain influence on the CTR. In this case, the weight parameters of the weighted grouping logic are adjusted based on the CTR deviation. For example, if the CTR of the red button is higher than that of the blue button, the weight parameters are adjusted to increase the probability of entering the red button, i.e., increasing the group traffic of group A. Furthermore, increasing the sample size further improves the authenticity of the test results. The conversion rate deviation of Group A after increasing traffic to Group A is calculated, i.e., the click-through rate (CTR) change of the red button is calculated. Based on the CTR change and the weight parameters of the grouping logic, a preset weighted calculation is performed to obtain confidence information. The confidence information represents the effectiveness of the grouping test; the higher the confidence, the greater the impact of the corresponding group on the conversion event. For example, a higher confidence for Group A indicates that the red button can improve the CTR. When the confidence for Group A exceeds a preset confidence threshold, a decision report on whether Group A can be launched is generated, and Group A is launched according to a preset release process. In other words, the red button will be used as the launch button color. This embodiment is based on real-time data-driven weight adjustment, avoiding manual intervention; and based on statistical testing, it ensures the reliability of A / B testing and reduces the risk of misjudgment; furthermore, the conclusions of A / B testing directly generate decisions and are released, shortening the iteration cycle.

[0067] A third aspect of the present invention provides a computer-readable storage medium comprising an A / B test gateway control method program based on adaptive flow splitting, wherein when the A / B test gateway control method program based on adaptive flow splitting is executed by a processor, it implements the steps of the A / B test gateway control method based on adaptive flow splitting as described in any of the preceding claims.

[0068] In summary, this invention provides an A / B testing gateway control method, system, and storage medium based on adaptive traffic splitting. First, traffic is grouped according to user requests using a preset weighted grouping logic to ensure uniform traffic distribution and user consistency. Second, the conversion rate sequence is dynamically calculated based on the number of conversion events triggered by a preset segmentation window. Then, when the conversion rate deviation exceeds a threshold, the grouping weights are dynamically adjusted using a preset relational formula to achieve adaptive incremental optimization. Finally, the conversion rate trend is verified after weight adjustment, and a confidence level is calculated using statistical tests. Once the confidence level meets the standard, a canary release process is automatically triggered. This invention integrates A / B testing into a closed-loop process of adaptive statistical adjustment, solving the problem of uneven traffic distribution and improving decision-making response efficiency.

[0069] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for controlling an A / B test gateway based on adaptive traffic splitting, characterized in that, The method includes: Receive and parse the user request to obtain the request ID; Based on the analysis of the request ID and the preset AB test ID pool, traffic grouping information is obtained through hash processing and combined with preset weight grouping logic; Based on the traffic grouping information, capture preset conversion events; Based on a preset segmentation window, the conversion rate is calculated according to the conversion event; Based on the test ID, query the conversion rate sequence and calculate the conversion deviation rate; Based on the analysis of the conversion deviation rate, the weight parameters of the weight grouping logic are dynamically adjusted; Calculate the conversion rate deviation before and after the weight parameters are adjusted, and combine it with the adjusted weight parameters to obtain the confidence information; Based on the analysis of confidence information and the generation of a decision report using the test ID, a test version is released according to traffic grouping information.

2. The A / B test gateway control method based on adaptive traffic splitting according to claim 1, characterized in that, Based on the analysis of the request ID and the preset A / B test ID pool, and combined with the preset weight grouping logic, traffic grouping information is obtained, specifically: Parse the user request to extract the request ID and test ID; Based on a preset hash algorithm, the request ID and test ID are merged and hashed to obtain a hash value; Take a consecutive byte substring from the hash value, calculate the ratio of the value of the consecutive byte substring to the maximum feasible value of the corresponding consecutive byte substring, and give the mapping value; Divide the grouping intervals according to the weight parameters of the weight grouping logic; Based on the grouping range to which the mapped value belongs, the user request is associated with the traffic group, and traffic group information is given, wherein the traffic group includes at least group A and group B.

3. The A / B test gateway control method based on adaptive traffic splitting according to claim 2, characterized in that, The request ID is extracted, and further includes: Based on the request ID, obtain the device type, user attributes, and geographic region information; Based on the timestamp, device type, population attributes, and geographical area information, label dimensions are defined, the encoding of each label dimension is determined, and the request ID is processed to form a multidimensional structured label.

4. The A / B test gateway control method based on adaptive traffic splitting according to claim 1, characterized in that, The step of capturing preset conversion events based on the traffic grouping information specifically includes: The traffic grouping information is matched with a predefined set of conversion event rules, wherein the set of conversion event rules includes feature interfaces, feature fields and feature parameters; If a conversion event request from the feature interface is received, the field parameters are obtained by parsing the feature fields. Determine whether the field parameter is the same as the feature parameter; If so, it means that a conversion event has been captured.

5. The A / B test gateway control method based on adaptive traffic splitting according to claim 1, characterized in that, The step of dynamically adjusting the weight parameters of the weight grouping logic based on the conversion deviation rate is specifically as follows: Obtain the first dynamic adjustment relation; Based on the first dynamic adjustment relationship, the first adjustment weight is obtained according to the conversion deviation rate; The first weight threshold is obtained based on the current weight parameters of the weight grouping logic; Based on the analysis of the first adjustment weight and the first weight threshold, the adjustment weight parameters of the weight grouping logic are given.

6. The A / B test gateway control method based on adaptive traffic splitting according to claim 5, characterized in that, The first dynamic adjustment relationship is specifically expressed as follows: ； Where, Δw A The adjustment magnitude of the weight parameters for group A, Δw B Here, K is the adjustment magnitude of the weight parameters for group B, and C is the step size parameter. A Let C be the conversion rate of group A. B The conversion rate for group B; Adjusting the weight parameters is specifically expressed as follows: ； Among them, w A,T w is the adjusted weighting parameter for group A. B,T For the adjusted weighting parameters of group B, w A,C Here are the current weight parameters for group A, where α is the upper limit of the weight ratio coefficient, and w max To adjust the upper limit of the weight, w B,C Here are the current weight parameters for group B, and min is the function to find the minimum value.

7. The A / B test gateway control method based on adaptive traffic splitting according to claim 1, characterized in that, The conversion rate deviation before and after the weight parameter adjustment is calculated, and combined with the adjusted weight parameter, the confidence information is obtained, specifically: Based on the conversion rate deviation, determine whether the conversion rate after adjusting the weight parameters shows a downward trend; If so, the confidence level information is set to a preset first confidence reference value; If not, then the first weight information is determined based on the conversion rate deviation; The confidence level information is calculated based on the first weight information and the weight parameters according to a preset weighting algorithm.

8. The A / B test gateway control method based on adaptive traffic splitting according to claim 7, characterized in that, Based on the first weight information and the weight parameters, the confidence information is calculated according to a preset weighting algorithm, specifically as follows: ； Where CI is the confidence level, min is the minimum function, max is the maximum function, and w T For the adjusted weight parameters, C T To adjust the conversion rate, C C To adjust the conversion rate before adjustment, M min M is the lower confidence level. max This represents the upper limit of confidence.

9. An A / B test gateway control system based on adaptive current splitting, characterized in that, The system includes a memory and a processor. The memory includes a program for an AB test gateway control method based on adaptive flow splitting. When the processor executes the program for the AB test gateway control method based on adaptive flow splitting, it performs the following steps: Receive and parse the user request to obtain the request ID; Based on the analysis of the request ID and the preset AB test ID pool, traffic grouping information is obtained through hash processing and combined with preset weight grouping logic; Based on the traffic grouping information, capture preset conversion events; Based on a preset segmentation window, the conversion rate is calculated according to the conversion event; Based on the test ID, query the conversion rate sequence and calculate the conversion deviation rate; Based on the analysis of the conversion deviation rate, the weight parameters of the weight grouping logic are dynamically adjusted; Calculate the conversion rate deviation before and after the weight parameters are adjusted, and combine it with the adjusted weight parameters to obtain the confidence information; Based on the analysis of confidence information and the generation of a decision report using the test ID, a test version is released according to traffic grouping information.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer-readable storage medium includes a program for an adaptive flow-based AB test gateway control method. When the program is executed by a processor, it implements the steps of the adaptive flow-based AB test gateway control method as described in any one of claims 1 to 6.

Images