AB test system of API gateway and constraint realization shunt flow framework

Abstract

The invention discloses an AB test system of an API gateway and a constraint realization shunt capacity framework, which is established based on a shunt strategy of conflict detection constraint and comprises the API gateway, a unified monitoring platform, a QTP automatic test platform, an OTP container platform and an AB test platform. According to the AB test system for the API gateway and the constraint implementation of the shunt flow framework, the flow is efficiently allocated to different environments through flow scheduling, existing resources are fully utilized for testing, and meanwhile, a large-scale service scene is supported, so that the test efficiency is improved, the test cost is reduced, the stability under the high concurrency condition is ensured, and the test efficiency is improved. And different customer requirements are met.

Description

Technical Field

[0001] The present invention relates to the field of AB testing, and in particular to an AB testing system for an API gateway and a constraint-based traffic distribution framework. Background Art

[0002] AB testing is the sum of a series of methodologies based on statistical principles, which uses reasonable traffic distribution to efficiently and accurately select solutions to maximize the value of local indicators. Although the statistical basis of AB testing has a century of history, it is still a huge challenge to build a correct and reliable A / B testing platform on a large scale. Not only must we deal with the dual challenges of spillover effects and small samples in the experimental design stage, but we must also balance experimental bias and variance to determine appropriate experimental units, grouping methods, and analysis methods to provide a reasonable experimental design. In addition, we must deal with various statistical traps such as variance calculation, P value calculation, multiple comparisons, confounding factors, and false negatives in the analysis stage. Therefore, obtaining high-quality results requires an understanding of experiments and statistics, which undoubtedly increases the threshold for experiments and makes it difficult to achieve the goal of anyone conducting experiments being able to draw credible conclusions.

[0003] With the rapid development of the Internet, various application systems and services have become an indispensable part of modern society. In order to ensure the high availability, high performance and high concurrency of these application systems and services, stress testing has become an indispensable part of the software development and operation and maintenance process. Therefore, there is an urgent need for a method to solve the problems of low testing efficiency, low quality, incomplete coverage and difficult results analysis.

[0004] In order to improve the parallelism of experiments and run multiple mutually exclusive experiments at the same time, a constraint-based traffic distribution framework was built to regulate and constrain how different experiments share and use traffic. Through traffic scheduling, traffic is efficiently distributed to different environments and existing resources are fully utilized for testing. At the same time, it supports large-scale business scenarios, thereby improving test efficiency, reducing test costs, ensuring stability under high concurrency, and meeting different needs. Summary of the invention

[0005] The purpose of the present invention is to propose an AB testing system for an API gateway and a constraint-based traffic distribution framework in response to the above-mentioned technical problems.

[0006] An AB testing system with an API gateway and constraint-based traffic diversion framework, established based on a traffic diversion strategy with conflict detection constraints, including an API gateway, a unified monitoring platform, a QTP automated testing platform, an OTP container platform, and an AB testing platform; The API gateway is the entrance to the AB testing platform, realizing traffic management allocation, experimental parameter configuration, traffic coloring, security authentication, version control, and grayscale release functions; The unified monitoring platform provides data support and monitoring means for AB testing; The QTP automated testing platform realizes the functions of requesting experimental configuration, simulating user behavior, and displaying experimental information; The OTP container platform includes an API gateway, a unified monitoring platform, a computing module, and a business service window; The AB testing platform selects a solution that maximizes the value of local indicators through traffic distribution.

[0007] Furthermore, an AB testing system with an API gateway and constraints to implement a traffic distribution framework, wherein the calculation module implements significance analysis, cohort analysis, indicator analysis, and sample size analysis functions; The business service window creates and executes a pipeline, connecting the service to a unified monitoring platform.

[0008] Furthermore, an AB testing system of an API gateway and a constraint-based traffic distribution framework is provided, wherein the traffic distribution strategy is: Traffic in different control groups of the same strategy is mutually exclusive; Different experiments between different strategies allow reuse of traffic if there are no common parameters between strategies.

[0009] Further, an AB testing system of an API gateway and a constraint-based traffic distribution framework, wherein the AB testing platform includes a test design and management module, a test execution and monitoring module, a data analysis and reporting module, an A / B / N test support module, an API publishing module, and a service orchestration module; The test design and management module realizes the functions of creating, editing, managing and deleting AB test projects and experimental grouping; The test execution and monitoring module automatically executes AB tests and monitors the test results in real time; The test results include conversion rate, click-through rate, and visit volume; The data analysis and reporting module performs test comparison, trend analysis, and funnel analysis on the AB test results, and generates test reports and data reports; The A / B / N test support module distributes traffic to different versions according to the strategy, collects user data, and compares the differences; The API publishing module publishes the API to the gateway and accesses the mock service through the API gateway; The service orchestration module combines multiple API interfaces published to the gateway into a composite API, and accesses the mock-orchestrated API through the API gateway.

[0010] Further, an AB testing system of an API gateway and a constraint-based traffic distribution framework, wherein the release of the API release module includes a real-time monitoring interface, a test data interface, and a business interface; The specific function of the API publishing module is to achieve traffic coloring and traffic control through the configuration of API policies.

[0011] Furthermore, an AB testing system of an API gateway and a constraint-based traffic distribution framework is provided, wherein the specific function of the service orchestration module is to allow the business party to perform visual process definition and integrate the single-function business process interface into an API with complete business logic; The process definition includes the link relationship of the orchestration service and the interface.

[0012] Furthermore, an AB testing system for an API gateway and a constraint-based traffic distribution framework, wherein the A / B / N testing support module includes a pre-setting submodule, a detection and reporting submodule. The pre-setting submodule creates the experiment, sets A / B / N variants, and sets related variables and targets; The detection and includes submodules to detect mobile phone experimental data, analyze experimental results, generate experimental reports, compare the performance of the experimental group and the control group, and obtain an analysis report; The mobile phone experiment data includes user access behavior, page click rate, and conversion rate.

[0013] Further, an AB testing system for an API gateway and a constraint-based traffic distribution framework, wherein the analysis report includes the cumulative number of users, page views per minute, traffic sources, users by time period, popular activity web pages, and regional distribution; The cumulative number of users reflects the user scale of the website or application and evaluates the popularity and market share of the website or application; The page views per minute reflect the number of visits and user activity of the website or application, and evaluate the traffic quality and user stickiness of the website or application; The traffic sources reflect the traffic sources and user access paths of the website or application, and evaluate the promotion effect and user acquisition channels of the website or application; The users divided by time period reflect the active time period of users of the website or application, and evaluate the access habits and behavioral characteristics of users in different time periods; The number of visits and user behavior of the popular activity pages of the popular activity webpage website to understand the user's attention and participation in the website activities; The geographical distribution reflects the geographical distribution of website users, and helps us understand the geographical distribution of website users and market coverage; The geographical distribution includes provinces and cities.

[0014] Beneficial effects of the present invention: (1) Constrained traffic diversion: Through gateway configuration parameters, fast traffic diversion can be achieved through weight and conditional routing, accurately directing different proportions of traffic to different versions of AB testing, supporting multivariate testing, and allocating users or traffic to different combination experimental groups to simultaneously evaluate the effects of multiple variables; (2) High experimental quality: Package and output the experimental design, lower the experimental threshold, ensure the experimental quality, and prevent participants from being influenced by other participants when participating in the test; (3) Traffic simulation: determine the traffic distribution and sample size of the experiment, evaluate the possible risks of the test, avoid deviations caused by traffic deviation, and help users find the best traffic distribution plan to improve the effect and efficiency of the experiment; (4) Flexible diversion: Flexibly set diversion strategies to allocate traffic more accurately to different experiments, reduce traffic waste, and increase experimental flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a structural diagram of an AB testing system that uses an API gateway and a constraint-based traffic distribution framework. DETAILED DESCRIPTION

[0016] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described with reference to the accompanying drawings.

[0017] As attached Figure 1 As shown, an AB testing system for an API gateway and a constraint-based traffic diversion framework is established based on a traffic diversion strategy based on conflict detection constraints, including an API gateway, a unified monitoring platform, a QTP automated testing platform, an OTP container platform, and an AB testing platform; The API gateway is the entrance to the AB testing platform, realizing traffic management allocation, experimental parameter configuration, traffic coloring, security authentication, version control, and grayscale release functions; The unified monitoring platform provides data support and monitoring means for AB testing; The QTP automated testing platform realizes the functions of requesting experimental configuration, simulating user behavior, and displaying experimental information; The OTP container platform includes an API gateway, a unified monitoring platform, a computing module, and a business service window; The AB testing platform selects a solution that maximizes the value of local indicators through traffic distribution.

[0018] Furthermore, an AB testing system with an API gateway and constraints to implement a traffic distribution framework, wherein the calculation module implements significance analysis, cohort analysis, indicator analysis, and sample size analysis functions; The business service window creates and executes a pipeline, connecting the service to a unified monitoring platform.

[0019] Furthermore, an AB testing system of an API gateway and a constraint-based traffic distribution framework is provided, wherein the traffic distribution strategy is: Traffic in different control groups of the same strategy is mutually exclusive; Different experiments between different strategies allow reuse of traffic if there are no common parameters between strategies.

[0020] Further, an AB testing system of an API gateway and a constraint-based traffic distribution framework, wherein the AB testing platform includes a test design and management module, a test execution and monitoring module, a data analysis and reporting module, an A / B / N test support module, an API publishing module, and a service orchestration module; The test design and management module realizes the functions of creating, editing, managing and deleting AB test projects and experimental grouping; The test execution and monitoring module automatically executes AB tests and monitors the test results in real time; The test results include conversion rate, click-through rate, and visit volume; The data analysis and reporting module performs test comparison, trend analysis, and funnel analysis on the AB test results, and generates test reports and data reports; The A / B / N test support module distributes traffic to different versions according to the strategy, collects user data, and compares the differences; The API publishing module publishes the API to the gateway and accesses the mock service through the API gateway; The service orchestration module combines multiple API interfaces published to the gateway into a composite API, and accesses the mock-orchestrated API through the API gateway.

[0021] Further, an AB testing system of an API gateway and a constraint-based traffic distribution framework, wherein the release of the API release module includes a real-time monitoring interface, a test data interface, and a business interface; The specific function of the API publishing module is to achieve traffic coloring and traffic control through the configuration of API policies.

[0022] Furthermore, an AB testing system of an API gateway and a constraint-based traffic distribution framework is provided, wherein the specific function of the service orchestration module is to allow the business party to perform visual process definition and integrate the single-function business process interface into an API with complete business logic; The process definition includes the link relationship of the orchestration service and the interface.

[0023] Furthermore, an AB testing system for an API gateway and a constraint-based traffic distribution framework, wherein the A / B / N testing support module includes a pre-setting submodule, a detection and reporting submodule. The pre-setting submodule creates the experiment, sets A / B / N variants, and sets related variables and targets; The detection and includes submodules to detect mobile phone experimental data, analyze experimental results, generate experimental reports, compare the performance of the experimental group and the control group, and obtain an analysis report; The mobile phone experiment data includes user access behavior, page click rate, and conversion rate.

[0024] Further, an AB testing system for an API gateway and a constraint-based traffic distribution framework, wherein the analysis report includes the cumulative number of users, page views per minute, traffic sources, users by time period, popular activity web pages, and regional distribution; The cumulative number of users reflects the user scale of the website or application and evaluates the popularity and market share of the website or application; The page views per minute reflect the number of visits and user activity of the website or application, and evaluate the traffic quality and user stickiness of the website or application; The traffic sources reflect the traffic sources and user access paths of the website or application, and evaluate the promotion effect and user acquisition channels of the website or application; The users divided by time period reflect the active time period of users of the website or application, and evaluate the access habits and behavioral characteristics of users in different time periods; The number of visits and user behavior of the popular activity pages of the popular activity webpage website to understand the user's attention and participation in the website activities; The geographical distribution reflects the geographical distribution of website users, and helps us understand the geographical distribution of website users and market coverage; The geographical distribution includes provinces and cities.

[0025] This solution uses an API gateway and constraints to implement an AB testing system with a traffic distribution framework. Through traffic scheduling, it efficiently distributes traffic to different environments and makes full use of existing resources for testing. It also supports large-scale business scenarios, thereby improving testing efficiency, reducing testing costs, ensuring stability under high concurrency, and meeting different customer needs.

[0026] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An AB testing system for an API gateway and a constraint-based traffic distribution framework, characterized in that: The diversion strategy based on conflict detection constraints includes API gateway, unified monitoring platform, QTP automated testing platform, OTP container platform, and AB testing platform; The API gateway is the entrance to the AB testing platform, realizing traffic management allocation, experimental parameter configuration, traffic coloring, security authentication, version control, and grayscale release functions; The unified monitoring platform provides data support and monitoring means for AB testing; The QTP automated testing platform realizes the functions of requesting experimental configuration, simulating user behavior, and displaying experimental information; The OTP container platform includes an API gateway, a unified monitoring platform, a computing module, and a business service window; The AB testing platform selects a solution that maximizes the value of local indicators through traffic distribution.

2. According to claim 1, an AB testing system for an API gateway and a constraint-based traffic distribution framework, characterized in that: The calculation module realizes the functions of significance analysis, cohort analysis, indicator analysis, and sample size analysis; The business service window creates and executes a pipeline, connecting the service to a unified monitoring platform.

3. According to claim 1, an AB testing system for an API gateway and a constraint-based traffic distribution framework, characterized in that: The diversion strategy is: Traffic in different control groups of the same strategy is mutually exclusive; Different experiments between different strategies allow reuse of traffic if there are no common parameters between strategies.

4. According to claim 1, an AB testing system for an API gateway and a constraint-based traffic distribution framework, characterized in that: The AB testing platform includes a test design and management module, a test execution and monitoring module, a data analysis and reporting module, an A / B / N test support module, an API publishing module, and a service orchestration module; The test design and management module realizes the functions of creating, editing, managing and deleting AB test projects and experimental grouping; The test execution and monitoring module automatically executes AB tests and monitors the test results in real time; The test results include conversion rate, click-through rate, and visit volume; The data analysis and reporting module performs test comparison, trend analysis, and funnel analysis on the AB test results, and generates test reports and data reports; The A / B / N test support module distributes traffic to different versions according to the strategy, collects user data, and compares the differences; The API publishing module publishes the API to the gateway and accesses the mock service through the API gateway; The service orchestration module combines multiple API interfaces published to the gateway into a composite API, and accesses the mock-orchestrated API through the API gateway.

5. According to claim 4, an AB testing system for an API gateway and a constraint-based traffic distribution framework is characterized in that: The release of the API release module includes real-time monitoring interface, test data interface, and business interface; The specific function of the API publishing module is to achieve traffic coloring and traffic control through the configuration of API policies.

6. According to claim 4, an AB testing system for an API gateway and a constraint-based traffic distribution framework, characterized in that: The specific function of the service orchestration module is to allow the business party to perform visual process definition and integrate the single-function business process interface into an API with complete business logic; The process definition includes the link relationship of the orchestration service and the interface.

7. According to claim 4, an AB testing system for an API gateway and a constraint-based traffic distribution framework, characterized in that: The A / B / N test support module includes a presetting submodule, a detection and reporting submodule The pre-setting submodule creates the experiment, sets A / B / N variants, and sets related variables and targets; The detection and includes submodules to detect mobile phone experimental data, analyze experimental results, generate experimental reports, compare the performance of the experimental group and the control group, and obtain an analysis report; The mobile phone experiment data includes user access behavior, page click rate, and conversion rate.

8. According to claim 7, an AB testing system for an API gateway and a constraint-based traffic distribution framework, characterized in that: The analysis report includes the cumulative number of users, page views per minute, traffic sources, users by time period, popular activity web pages, and geographical distribution; The cumulative number of users reflects the user scale of the website or application and evaluates the popularity and market share of the website or application; The page views per minute reflect the number of visits and user activity of the website or application, and evaluate the traffic quality and user stickiness of the website or application; The traffic sources reflect the traffic sources and user access paths of the website or application, and evaluate the promotion effect and user acquisition channels of the website or application; The users divided by time period reflect the active time period of users of the website or application, and evaluate the access habits and behavioral characteristics of users in different time periods; The number of visits and user behavior of the popular activity pages of the popular activity webpage website to understand the user's attention and participation in the website activities; The geographical distribution reflects the geographical distribution of website users, and helps us understand the geographical distribution of website users and market coverage; The geographical distribution includes provinces and cities.

Images