AB test method, device and equipment in travel order distribution field and storage medium

Abstract

The embodiment of the invention provides an AB test method, device and equipment in the field of travel order distribution and a storage medium, and relates to the technical field of experimental testing. The method comprises the following steps: screening out a target historical fragment meeting a preset similar feature condition, and determining a future time slice set; distributing the future time slices to construct a plurality of synthetic control groups; in a future test stage, shunting the actual car use order flow according to the time-space attribute, so that each synthetic control group executes the order dispatching operation on the shunted order flow based on the respective order dispatching strategy; and calculating the key business index of each synthetic control group in the process of executing the order dispatching operation in real time. According to the embodiment of the invention, the time slices with similar characteristics are screened and the synthetic control group is constructed, and then the order flow is shunted according to the time-space attributes in the experiment process and is allocated to the synthetic control group to execute the order dispatching operation, so that the accuracy of AB testing in the travel order dispatching field is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of experimental testing technology, and more specifically, to an AB testing method, device, equipment and storage medium in the field of travel dispatching. Background Art

[0002] In the field of travel services, a reasonable dispatching strategy can effectively improve the order matching rate, reduce the waiting time of passengers, and improve the efficiency of drivers in accepting orders, thereby improving the overall operational efficiency and service quality, and providing users with a better travel experience. With the development of the market and the advancement of technology, continuously optimizing the dispatching strategy has become one of the key tasks of major travel platforms.

[0003] As a scientific experimental method, AB testing is widely used in product iteration and function optimization. In the travel industry, AB testing can be used to compare the effects of different dispatching strategies, evaluate the actual performance of new strategies, and thus guide the optimization direction of dispatching strategies.

[0004] At present, in the complex scenario of travel dispatching involving multi-dimensional influencing factors such as time and space, the existing AB testing scheme that simply diverts traffic is difficult to isolate potential contamination between the experimental group and the control group, resulting in low test accuracy. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an AB testing method, device, equipment and storage medium in the field of travel dispatching, so as to improve the accuracy of AB testing in the field of travel dispatching.

[0006] In a first aspect, an embodiment of the present application provides an AB testing method in the field of travel dispatching, including:

[0007] Divide the historical period into a number of historical segments according to a preset unit time period, select target historical segments that meet preset similar feature conditions from the several historical segments based on the historical order data corresponding to the historical period, and determine a future time slice set corresponding to a future test phase based on each of the target historical segments; wherein the future time slices in the future time slice set correspond to each of the target historical segments one by one;

[0008] Allocating the future time slices in the future time slice set to obtain a plurality of synthetic control groups corresponding to the future test phase;

[0009] During the future test phase, the actual vehicle order flow generated in real time is divided according to the spatiotemporal attributes, and the corresponding actual vehicle order flow after the division is allocated to the corresponding synthetic control group, so that each synthetic control group performs the dispatching operation based on its own dispatching strategy; wherein the dispatching strategy includes at least two;

[0010] During the future trial phase, key business indicators of each synthetic control group in the process of performing dispatch operations are calculated in real time.

[0011] In an embodiment of the present application, time slices with similar characteristics are screened based on historical data to predict and construct a synthetic control group for the future experimental stage. During the experiment, the actual order flow is diverted according to the time and space attributes and allocated to the corresponding synthetic control group to perform the dispatching operation, thereby effectively isolating the potential contamination factors between different experimental groups and effectively improving the accuracy of AB testing in the field of travel dispatching.

[0012] In some possible embodiments, after the real-time calculation of the key business indicators of each synthetic control group in the process of executing the dispatching operation, the method further includes:

[0013] During the future test phase, a preset multi-armed bandit algorithm is used to dynamically adjust the allocation of future time slices not performed by each synthetic control group based on each of the key business indicators.

[0014] In an embodiment of the present application, by utilizing a multi-armed bandit algorithm to dynamically adjust the resource allocation of each experimental group based on experimental indicators, it is possible to ensure effective utilization of resources and further improve the efficiency and accuracy of AB testing in the field of travel dispatching.

[0015] In some possible embodiments, the step of selecting a target historical segment that meets a preset similar feature condition from a plurality of historical segments based on the historical order data corresponding to the historical period includes:

[0016] Determine basic feature information corresponding to each of the historical segments based on the historical order data; wherein the basic feature information includes at least one of weather features, holiday features, and traffic conditions features;

[0017] A target historical segment whose basic feature information matches the preset basic feature is selected from each of the historical segments.

[0018] In an embodiment of the present application, by screening out time slice combinations with similar characteristics based on the matching of basic feature information, the interference of external factors on the experimental results can be effectively reduced, further improving the efficiency and accuracy of AB testing in the field of travel dispatching.

[0019] In some possible embodiments, the step of selecting a target historical segment that meets a preset similar feature condition from a plurality of historical segments based on the historical order data corresponding to the historical period includes:

[0020] Determine basic feature information and vehicle demand feature information corresponding to each of the historical segments based on the historical order data; wherein the basic feature information includes at least one of weather features, holiday features, and traffic conditions, and the vehicle demand feature information includes at least one of vehicle order distribution features, vehicle supply features, and vehicle distribution features;

[0021] A candidate historical segment whose basic feature information matches the preset basic feature is screened out from each of the historical segments, and a target historical segment whose vehicle use demand feature information matches the preset vehicle use demand feature is screened out from each of the candidate historical segments.

[0022] In an embodiment of the present application, by first screening the time segments based on the matching of basic feature information, and then screening the time segments based on the similarity of vehicle demand characteristics, the interference of potential factors on the experimental results can be further reduced, thereby further improving the efficiency and accuracy of AB testing in the field of travel dispatching.

[0023] In some possible embodiments, the AB testing method in the field of travel dispatching further includes:

[0024] Determine feature prediction information corresponding to each of the future time slices based on the historical order data; wherein the feature prediction information includes at least one of basic feature prediction information and vehicle demand feature prediction information, the basic feature prediction information includes at least one of weather features, holiday features, and traffic conditions, and the vehicle demand feature prediction information includes at least one of vehicle order distribution features, vehicle supply features, and vehicle distribution features;

[0025] During the future test phase, actual feature monitoring information corresponding to each future time slice is acquired in real time; wherein the type of the actual feature monitoring information corresponds to the feature prediction information;

[0026] When it is determined that the error between the feature prediction information and the actual feature monitoring information corresponding to the same future time slice exceeds a preset fault tolerance threshold, the experimental emergency adjustment strategy is executed; wherein, the experimental emergency adjustment strategy includes interrupting the experiment of the synthetic control group corresponding to this future time slice.

[0027] In an embodiment of the present application, by comparing the actual monitoring characteristics of each time slice with the corresponding predicted characteristics, and taking timely response strategies for the experimental group when the comparison error exceeds the allowable range, the reliability and accuracy of AB testing in the field of travel dispatch can be further improved.

[0028] In some possible embodiments, the actual vehicle order flow generated in real time is divided according to the time and space attributes, and the corresponding actual vehicle order flow after the division is allocated to the corresponding synthetic control group, including:

[0029] Based on the location information and time information of the actual car order generated in real time, the actual car order flow is mapped to a number of independent flow pools based on geographical regions according to a preset bucketing strategy, and at the same time, the actual car order flow in each independent flow pool is layered according to the unit time period;

[0030] A target independent traffic pool matching the geographical area of ​​each of the synthetic control groups is determined, and the actual vehicle order traffic matching each of the future time slices in the target independent traffic pool is allocated to the corresponding synthetic control group.

[0031] In an embodiment of the present application, by first bucketing the actual order flow according to the geographical area and then stratifying the actual order flow according to the division of time slices, the orthogonality and purity between the experiments can be guaranteed, thereby further improving the reliability and accuracy of AB testing in the field of travel dispatching.

[0032] In a second aspect, an embodiment of the present application provides an AB testing device in the field of travel dispatching, including:

[0033] A time slice screening module, used to divide the historical period into a number of historical segments according to a preset unit time period, screen out target historical segments that meet preset similar feature conditions from the several historical segments based on the historical order data corresponding to the historical period, and determine a future time slice set corresponding to the future test stage based on each of the target historical segments; wherein the future time slices in the future time slice set correspond to each of the target historical segments one by one;

[0034] A control group construction module, used for allocating future time slices in the future time slice set to obtain a plurality of synthetic control groups corresponding to the future test phase;

[0035] A diversion and dispatching module is used to divert the actual vehicle order flow generated in real time according to the spatiotemporal attributes during the future test phase, and distribute the corresponding actual vehicle order flow after diversion to the corresponding synthetic control group, so that each synthetic control group performs the dispatching operation based on its own dispatching strategy; wherein the dispatching strategy includes at least two;

[0036] The indicator calculation module is used to calculate the key business indicators of each synthetic control group in the process of executing the dispatching operation in real time during the future trial stage.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the method described in any embodiment of the first aspect when executing the program.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in any embodiment of the first aspect can be implemented.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, wherein when the computer program is executed by a processor, the method described in any embodiment of the first aspect can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flow chart of an AB testing method in the field of travel dispatching provided in an embodiment of the present application;

[0042] Figure 2 AB test and strategy optimization iteration diagram provided in the embodiment of the present application;

[0043] Figure 3 A schematic diagram of the structure of an AB testing device in the field of travel dispatching provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0046] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0047] It should be noted that in the travel industry, the efficiency analysis of dispatching operation strategies is usually conducted through AB testing and user feedback surveys. Among them, the AB testing program randomly divides the user group into several groups, applies different dispatching strategies to each group, and directly compares the performance of each group on key business indicators, so as to scientifically evaluate the effectiveness of the new strategy; the user feedback survey program directly collects service experience feedback from users, which is an important way to obtain users' real needs and preferences. Through questionnaires and other methods, companies can deeply understand users' views on new functions and new strategies, collect valuable opinions, and provide direction for subsequent optimization.

[0048] However, traditional AB testing schemes often require a relatively large sample size, which undoubtedly prolongs the experimental cycle and increases the resource investment in design, execution and monitoring. Although the principle of random allocation of traffic is intended to minimize deviations, in the complex scenario of travel dispatch involving multi-dimensional influencing factors of time and space, simple random diversion of traffic is difficult to completely isolate the potential contamination between the experimental group and the control group, thus affecting the accuracy of the test. In addition, although it is relatively direct to use user feedback surveys as the basis for evaluating the effectiveness of strategies, this method is easily affected by subjective bias, and the evaluation standards and expectations of different users vary significantly. In addition, the lag of feedback surveys may obscure the feedback of the immediate effect of the strategy, and the complexity of multiple factors requires in-depth analysis of the interaction between variables, which increases the difficulty of analysis and leads to unsatisfactory efficiency and effect of strategy optimization.

[0049] To this end, an embodiment of the present application provides an AB testing method in the field of travel dispatching, which aims to achieve pollution-free and efficient analysis and processing of dispatching strategies in a small sample size environment, improve the efficiency and accuracy of AB testing, and thus better guide the improvement of the overall effectiveness and benefits of travel dispatching strategies.

[0050] like Figure 1 As shown, the embodiment of the present application provides an AB testing method in the field of travel dispatching, which may include the steps of:

[0051] S1. Divide the historical period into several historical segments according to the preset unit time period, select the target historical segments that meet the preset similar feature conditions from the several historical segments based on the historical order data corresponding to the historical period, and determine the future time slice set corresponding to the future test stage based on each target historical segment; wherein the future time slices in the future time slice set correspond to each target historical segment one by one.

[0052] Specifically, first obtain the historical order data of a preset historical period. The preset historical period can be set according to demand, such as the past month, one week, etc.; then divide the historical period into several historical segments based on the preset unit time period. For example, if the unit time period is set to one hour, the past week (historical period) can be divided into 7×24=168 historical segments.

[0053] Historical order data includes the vehicle demand situation in historical periods and corresponding characteristic information, and this information is divided according to unit time periods. For example, the vehicle demand situation can be the number of vehicle demand orders corresponding to each historical segment, and the characteristic information can include weather characteristics, holiday characteristics (working days / holidays, working hours / rest hours) and traffic conditions characteristics (congestion conditions) corresponding to each historical segment.

[0054] Specifically, a typical reference feature range can be determined according to the test requirements. For example, a certain historical segment is selected as a reference, and then the vehicle demand and feature information of the reference historical segment are used as a benchmark. Historical segments with a certain degree of similarity (the features are the same or the similarity is within a preset error tolerance range) with the vehicle demand and feature information of the reference historical segment are screened out from all historical segments as target historical segments, to obtain a set of future time slices corresponding to the future test phase. For example, the historical segment of 9-10 am on Monday can be selected as a reference. Assuming that according to the historical order data, it can be determined that the vehicle demand of the historical segment is 100,000 orders, the traffic condition characteristics are 50% (calculated according to the preset rules based on the urban traffic heat map), the holiday characteristics are the working hours on weekdays, and the weather characteristics are sunny; then, based on these reference features, all historical segments can be screened according to the historical order data, among which the vehicle demand screened as the target historical segment needs to be around 100,000 orders (for example, 98,000-102,000 orders), the traffic condition characteristics are around 50% (for example, 45%-55%), the holiday characteristics are also the working hours on weekdays, and the weather characteristics are sunny. It should be noted that the same or similar judgment rules for each feature can be set according to demand.

[0055] It is understandable that the demand for cars under the same traffic conditions, weather, holidays and peak hours fluctuates within a relatively stable statistical range, and this fluctuation range is very small. Based on this, according to the above-mentioned selected target historical segments, non-overlapping time segments with the same (or similar) traffic conditions, weather, holidays and peak period characteristics can be selected from the future time period data. Therefore, a set of predicted time period members (future time slice set) can be constructed, and these future time slices show statistical similarities in characteristics such as car demand, which can be used as the basic unit for constructing a synthetic control group. It should be noted that each predicted future time slice is in a one-to-one correspondence with the selected target historical segment. For example, when the target historical segment selected based on the historical order data includes two time periods of 9-10 am on Monday and 6-7 pm on Wednesday, the corresponding future time slices correspondingly include two time periods of 9-10 am on Monday and 6-7 pm on Wednesday in the future test phase.

[0056] It should be noted that in addition to the feature types and similarity judgment rules illustrated in the above examples, more types of features can be introduced and different similarity judgment rules can be set according to actual scenarios, so as to further improve the similarity of time slice members between each synthetic control group, making the experimental results more accurate and closer to the actual car usage scenario.

[0057] S2. Allocate the future time slices in the future time slice set to obtain a number of synthetic control groups corresponding to the future test stages.

[0058] Specifically, it can be pre-constructed and divided into multiple synthetic control groups according to the test requirements. For example, if there are two order dispatching strategies that need to be tested (Strategy A and Strategy B), 10 synthetic control groups can be constructed, of which 5 synthetic control groups are used to execute Strategy A and the other 5 synthetic control groups are used to execute Strategy B; or 3 synthetic control groups can be used to execute Strategy A and the other 7 synthetic control groups can be used to execute Strategy B, etc. The specific ratio can also be set according to the requirements.

[0059] For the constructed synthetic control group, each future time slice can be allocated to a different synthetic control group from the selected future time slice set according to a preset allocation rule. Exemplarily, the allocation rule can follow the average principle and the random allocation principle, that is, for each future time slice, there is an equal probability (1 / 10) of being allocated to any one of the 10 synthetic control groups, and finally each synthetic control group will cover a substantially equal number of future time slices as a whole.

[0060] S3. During the future test phase, the actual vehicle order flow generated in real time will be diverted according to the time and space attributes, and the corresponding actual vehicle order flow after diversion will be allocated to the corresponding synthetic control group, so that each synthetic control group performs the dispatch operation based on its own dispatch strategy; wherein the dispatch strategy includes at least two.

[0061] It should be noted that in the future test phase, which is the phase of dispatching and matching real order traffic, the travel service platform will divert the actual car order traffic (real taxi orders) generated in real time according to the spatiotemporal attributes, that is, divide it into different traffic pools according to the location and time of each order. Then, according to the location area corresponding to each synthetic control group and the time period corresponding to each future time slice in each synthetic control group, each traffic pool is mapped with the corresponding synthetic control group (corresponding to the same location area and the same time period). Based on this, the actual car order traffic after diversion will be assigned to the corresponding synthetic control group, and dispatch operations will be performed on the order traffic obtained in each synthetic control group.

[0062] S4. During the future trial phase, the key business indicators of each synthetic control group in the process of executing the dispatching operation are calculated in real time.

[0063] Specifically, during the future trial phase, the experimental data coloring strategy can be used to mark and distinguish the relevant data, and the key business indicators of each synthetic control group in the process of executing the dispatching operation can be calculated based on these marks. For example, the key business indicators may include platform ROI (return on investment), driver income growth, and user experience satisfaction, etc. The specific indicator type and calculation method can be set according to demand.

[0064] Based on this, the performance of different dispatch strategies can be evaluated according to the key business indicators corresponding to each synthetic control group, so that the dispatch strategy can be optimized and adjusted according to these performances, or the traffic ratio of different synthetic control groups can be adjusted in real time according to these changes, so as to optimize resource allocation and improve the efficiency of travel dispatch services.

[0065] It should be noted that the embodiment of the present application screens time slices with similar characteristics based on historical order data to predict and construct a synthetic control group for the future experimental stage, and then diverts the actual order flow according to the time and space attributes and allocates it to the corresponding synthetic control group to perform the dispatching operation during the experiment, thereby effectively isolating the potential contamination factors between different experimental groups and avoiding the interference of irrelevant factors on the experimental results, thereby effectively improving the accuracy of AB testing in the field of travel dispatching.

[0066] In some possible embodiments, after step S4, the following steps may also be included:

[0067] During the future trial phase, the preset multi-armed bandit algorithm is used to dynamically adjust the allocation of future time slices not performed by each synthetic control group based on key business indicators.

[0068] It should be noted that as an advanced online learning algorithm, the Multi-Armed Bandit (MAB) algorithm can find the best balance between continuous exploration and utilization of existing knowledge, and achieve dynamic optimization of strategies and maximize benefits. By introducing the multi-armed bandit algorithm, its intelligent decision-making mechanism can dynamically adjust resource allocation, give priority to exploring potential high-yield strategy branches, and ensure the effective use of resources, thereby achieving the optimal balance between cost and benefit.

[0069] Specifically, first select the appropriate MAB algorithm based on the experimental settings and objectives. In current industry practices, four mainstream strategies have evolved in the field of multi-armed bandit algorithms, namely: naive random selection, the classic ε-Greedy strategy, the probability-based Boltzmann exploration method, and the Upper ConfidenceBound (UCB) strategy, which is known for its stability and reliability. Given the high requirements of the travel industry for result stability and optimization efficiency, the use of the UCB strategy has become the industry's preferred solution. After selecting a suitable MAB algorithm, you need to initialize the model parameters, such as the exploration rate ε, the confidence interval width, and so on.

[0070] Then, during the experimental operation phase (during the future trial phase), relevant data indicators (key business indicators) of each synthetic control group are collected in real time. These data can be preprocessed, including cleaning, sorting and analysis, to ensure the accuracy and reliability of the data. Then, the MAB algorithm is used to dynamically adjust the allocation ratio of resources to different synthetic control groups based on these indicator data. Specifically, the MAB algorithm can evaluate the performance of each synthetic control group in executing its dispatching strategy based on indicators such as platform ROI, driver income growth, and user experience satisfaction. For synthetic control groups with better performance, its resource (future time slices that have not been carried out) allocation ratio is increased; for synthetic control groups with poor performance, its resource allocation ratio is reduced or the allocation is stopped.

[0071] It is understandable that, since the experiment of a synthetic control group ends when the future time slice of the synthetic control group is executed, the experimental process of the corresponding synthetic control group will increase or decrease as the future time slices allocated to each synthetic control group increase or decrease. Based on this, for a synthetic control group with poor performance (usually indicating that the corresponding dispatching strategy is not effective as expected), the experiment of the synthetic control group can be terminated as soon as possible or immediately (stop the dispatching operation by the synthetic control group); and for a synthetic control group with good performance (usually indicating that the corresponding dispatching strategy is effective), the experiment of the synthetic control group can be extended and maintained (keep the dispatching operation by the synthetic control group) by increasing the subsequent allocation ratio of future time slices.

[0072] Based on this, by repeating the above process, the experimental strategy can be continuously iterated and optimized. In addition, during the experiment, the parameters of the MAB algorithm can be adjusted or the algorithm type can be switched as needed to obtain better experimental results.

[0073] It should be noted that if Figure 2 As shown in the figure, A / B testing constitutes a continuously evolving "learning cycle", which begins with the formulation of hypotheses, clearly defines the quantitative criteria for success, and then implements tests (i.e., A / B experiments) to verify these hypotheses. Through meticulous data analysis, valuable experience is learned, and the strategy is adjusted and implemented accordingly. This process not only marks the completion of the current cycle, but also the starting point for a new round of hypothesis construction, thus forming a seamless closed-loop system. In multiple rounds of repeated experimental iterations, the strategy is continuously optimized and gradually approaches the optimal state.

[0074] The embodiment of the present application first utilizes the rigor of AB testing, and by comparing the performance of different strategies in a real environment, provides an intuitive and quantitative basis for decision-making. In addition, by combining the advantages of the synthetic control method, by constructing a highly simulated control group, the reliability and robustness of the experimental results are further enhanced, and the interference of external factors on the experimental results is effectively reduced. On this basis, the multi-armed tiger machine algorithm is introduced, and its intelligent decision-making mechanism can dynamically adjust resource allocation, give priority to exploring potential high-yield strategy branches, and ensure the effective use of resources at the same time, thereby achieving the optimal balance between cost and benefit. In addition, the embodiment of the present application also particularly emphasizes the importance of spatiotemporal attributes. By considering the impact of different time periods and geographical locations on the experimental results, the dynamic characteristics of market changes and user behavior can be captured more accurately, so that the experimental conclusions are closer to the actual situation, and the applicability and effectiveness of the strategy are improved.

[0075] In some possible embodiments, in step S1, selecting a target historical segment that meets a preset similar feature condition from a plurality of historical segments based on the historical order data corresponding to the historical period may include:

[0076] S101, determining basic feature information corresponding to each historical segment based on historical order data; wherein the basic feature information includes at least one of weather features, holiday features, and traffic condition features;

[0077] S102: Filter out target historical segments whose basic feature information matches preset basic features from various historical segments.

[0078] It should be noted that urban traffic heat maps can be used to understand traffic flows at different times, helping to predict how traffic congestion affects car demand; weather forecast data can reflect the impact of weather changes on people's travel choices, thus helping to better understand how weather factors drive changes in car demand; holiday behavior patterns reveal people's travel habits and preferences during specific periods of time, helping to identify fluctuations in car demand during holidays.

[0079] By integrating the above multi-dimensional factors into the synthetic control model, the vehicle demand in the target area without external intervention can be simulated and predicted more accurately, thereby providing a more precise basis for evaluating the actual effect of any intervention measures.

[0080] Specifically, based on the historical order data, the basic feature information corresponding to each historical segment can be determined, including one or more of weather characteristics, holiday characteristics and traffic condition characteristics; then, the preset basic features are determined according to the test requirements, for example, the preset basic features are set to: the weather feature is sunny, the holiday feature is weekdays, and the traffic condition feature is busy; then, based on the basic feature information corresponding to other historical segments, the target historical segments with matching basic feature information can be screened out.

[0081] It can be understood that the matching of the basic feature information and the preset basic feature can indicate that the basic feature information and the preset basic feature are completely identical; or it can indicate that the basic feature information and the preset basic feature have a certain degree of similarity.

[0082] In some possible embodiments, in step S1, selecting a target historical segment that meets a preset similar feature condition from a plurality of historical segments based on the historical order data corresponding to the historical period may include:

[0083] S103, determining basic feature information and vehicle demand feature information corresponding to each historical segment based on the historical order data; wherein the basic feature information includes at least one of weather features, holiday features, and traffic conditions features, and the vehicle demand feature information includes at least one of vehicle order distribution features, vehicle supply features, and vehicle distribution features;

[0084] S104: Filter out candidate historical segments whose basic feature information matches preset basic features from each historical segment, and filter out target historical segments whose vehicle demand feature information matches preset vehicle demand features from each candidate historical segment.

[0085] It should be noted that, in addition to filtering the target historical segments only by basic feature information, the target historical segments can also be filtered by combining basic feature information and vehicle demand feature information. Specifically, the historical segments are first preliminarily screened based on the similarity of the basic feature information to obtain candidate historical segments, and then further screened according to the vehicle demand feature information corresponding to these candidate historical segments to obtain the final target historical segments. Among them, the vehicle demand feature information may include at least one of the vehicle order distribution feature, vehicle supply feature, and vehicle distribution feature.

[0086] It can be understood that, similarly, the matching of the vehicle use demand characteristic information with the preset vehicle use demand characteristic can indicate that the vehicle use demand characteristic information is exactly the same as the preset vehicle use demand characteristic; it can also indicate that the vehicle use demand characteristic information and the preset vehicle use demand characteristic have a certain similarity (the similarity is within a preset allowable range).

[0087] It should be noted that the feature types and quantities of the basic feature information and the vehicle demand feature information can be set according to the experimental requirements, and the similarity calculation method of each feature and the similarity threshold determination rule can be set according to the specific experimental requirements.

[0088] In some possible embodiments, the AB testing method in the field of travel dispatching may also include:

[0089] Determine feature prediction information corresponding to each future time slice based on historical order data; wherein the feature prediction information includes at least one of basic feature prediction information and vehicle demand feature prediction information, the basic feature prediction information includes at least one of weather features, holiday features, and traffic conditions, and the vehicle demand feature prediction information includes at least one of vehicle order distribution features, vehicle supply features, and vehicle distribution features;

[0090] During the future test phase, actual feature monitoring information corresponding to each future time slice is obtained in real time; wherein the type of the actual feature monitoring information corresponds to the feature prediction information;

[0091] When it is determined that the error between the feature prediction information and the actual feature monitoring information corresponding to the same future time slice exceeds a preset fault tolerance threshold, the experimental emergency adjustment strategy is executed; wherein the experimental emergency adjustment strategy includes interrupting the experiment of the synthetic control group corresponding to this future time slice.

[0092] It should be noted that since the future time slices of each synthetic control group are predicted and screened based on historical order data, in order to improve the reliability and accuracy of the experimental results, a comparison can be made based on the feature prediction information of each future time slice and the real feature information (actual feature monitoring information) collected in real time during the experiment. When there is a significant difference between the two, corresponding emergency adjustment strategies can be taken in a timely manner.

[0093] Specifically, it can be known from step S1 that the future time slices correspond to the target historical segments one by one, so the feature prediction information of each future time slice can be predicted based on the feature information of the corresponding target historical segment, or the feature information of the corresponding target historical segment can be directly used as the feature prediction information of the corresponding future time slice. It is understandable that the feature information of each target historical segment can be directly obtained based on the historical order data.

[0094] Then, during the future test phase, the actual feature monitoring information corresponding to each future time slice can be obtained based on the same dimension as the feature prediction information (consistent feature type and quantity), and then the error calculation is performed for the feature prediction information and the actual feature monitoring information corresponding to the same future time slice. Exemplarily, the total error of the feature prediction information and the actual feature monitoring information can be calculated by weighted summing the errors between each feature prediction value and the actual value.

[0095] When it is determined that the total error corresponding to a certain future time slice exceeds the preset fault tolerance threshold, the preset experimental emergency adjustment strategy can be executed. For example, the experiment of the synthetic control group covering this future time slice is interrupted to prevent the further spread of the deviation, and the other future time slices not conducted by the synthetic control group are allocated to other synthetic control groups to continue the experiment. In addition, the strategy for time segment screening in subsequent experiments and the parameters and configuration of each model can also be adjusted according to the error situation, so as to continuously optimize the efficiency of AB testing and the accuracy of experimental results.

[0096] In some possible embodiments, in step S3, the actual vehicle order flow generated in real time is divided according to the time and space attributes, and the corresponding actual vehicle order flow after the division is allocated to the corresponding synthetic control group, which may include:

[0097] S301, based on the location information and time information of the actual car order generated in real time, the actual car order flow is mapped to a number of independent flow pools based on geographical regions according to a preset bucketing strategy, and at the same time, the actual car order flow in each independent flow pool is layered according to a unit time period;

[0098] S302, determining a target independent traffic pool that matches the geographical area of ​​each synthetic control group, and allocating the actual vehicle order flow that matches each future time slice in the target independent traffic pool to the corresponding synthetic control group.

[0099] Specifically, the preset bucketing strategy can be constructed by combining multiple algorithms such as hash bucketing, regional bucketing, H3 bucketing, and Geohash bucketing. Through traffic bucketing, the actual car order traffic can be allocated to the corresponding independent traffic pools, and each independent traffic pool corresponds to the actual car order traffic in a geographical area; at the same time, for each independent traffic pool, the actual car order traffic therein can also be layered according to the unit time period. Since the length of the future time slice is screened and predicted based on the preset unit time period, the actual car order traffic of each layer will correspond to the time length of a future time slice.

[0100] Based on this, the actual vehicle order flow that has established a spatial and temporal mapping relationship with the flow pool can be allocated to the corresponding synthetic control group, so that the synthetic control group with corresponding spatiotemporal attributes can perform dispatch operations on these actual vehicle order flows.

[0101] It is understandable that in the same experiment, usually only synthetic control groups are set for the same area, so there may be multiple synthetic control groups that are mapped to the same independent flow pool. In addition, since the future time slices do not overlap with each other, each future time slice will only be mapped to one layer.

[0102] Please refer to Figure 3 , Figure 3 The block diagram of the AB testing device in the field of travel dispatching provided by some embodiments of the present application is shown. It should be understood that the AB testing device in the field of travel dispatching is similar to the above-mentioned Figure 1 Corresponding to the method embodiment, it is able to execute each step involved in the above method embodiment. The specific functions of the AB testing device in the travel dispatching field can be found in the description above. To avoid repetition, the detailed description is appropriately omitted here.

[0103] Figure 3 The AB testing device in the field of travel dispatching includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the AB testing device in the field of travel dispatching, and the AB testing device in the field of travel dispatching includes:

[0104] The time slice screening module 310 is used to divide the historical period into a number of historical segments according to a preset unit time period, screen out target historical segments that meet preset similar feature conditions from the several historical segments based on the historical order data corresponding to the historical period, and determine a future time slice set corresponding to the future test stage based on each target historical segment; wherein the future time slices in the future time slice set correspond to each target historical segment one by one;

[0105] A control group construction module 320 is used to allocate future time slices in the future time slice set to obtain a number of synthetic control groups corresponding to future test stages;

[0106] The diversion and dispatching module 330 is used to divert the actual vehicle order flow generated in real time according to the spatiotemporal attributes during the future test phase, and distribute the diverted corresponding actual vehicle order flow to the corresponding synthetic control group, so that each synthetic control group performs the dispatching operation based on its own dispatching strategy; wherein the dispatching strategy includes at least two;

[0107] The indicator calculation module 340 is used to calculate the key business indicators of each synthetic control group in the process of executing the dispatching operation in real time during the future trial phase.

[0108] It can be understood that the above-mentioned device item embodiment corresponds to the method item embodiment of the present invention. The AB testing device in the field of travel dispatch provided by the embodiment of the present invention can implement the AB testing method in the field of travel dispatch provided by any method item embodiment of the present invention.

[0109] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0110] like Figure 4 As shown, some embodiments of the present application provide an electronic device 400, which includes: a memory 410, a processor 420, and a computer program stored in the memory 410 and executable on the processor 420, wherein the processor 420 reads the program from the memory 410 through a bus 430 and executes the program to implement a method of any embodiment included in the above-mentioned AB testing method in the field of travel dispatching.

[0111] Processor 420 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 420 can be a microprocessor.

[0112] The memory 410 may be used to store instructions executed by the processor 420 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some or all functions of one or more modules described in the embodiments of the present application. The processor 420 of the disclosed embodiment may be used to execute instructions in the memory 410 to implement the method shown above. The memory 410 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.

[0113] Some embodiments of the present application further provide a computer-readable storage medium having a computer program stored thereon. The computer program is executed by a processor to execute the method described in the method embodiment.

[0114] Some embodiments of the present application further provide a computer program product, which, when executed on a computer, enables the computer to execute the method described in the method embodiment.

[0115] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0116] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0118] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0119] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0120] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0121] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. An AB testing method in the field of travel dispatching, characterized in that: include: Divide the historical period into a number of historical segments according to a preset unit time period, select target historical segments that meet preset similar feature conditions from the several historical segments based on the historical order data corresponding to the historical period, and determine a future time slice set corresponding to a future test phase based on each of the target historical segments; wherein the future time slices in the future time slice set correspond to each of the target historical segments one by one; Allocating the future time slices in the future time slice set to obtain a plurality of synthetic control groups corresponding to the future test phase; During the future test phase, the actual vehicle order flow generated in real time is divided according to the spatiotemporal attributes, and the corresponding actual vehicle order flow after the division is allocated to the corresponding synthetic control group, so that each synthetic control group performs the dispatching operation based on its own dispatching strategy; wherein the dispatching strategy includes at least two; During the future trial phase, key business indicators of each synthetic control group in the process of performing dispatch operations are calculated in real time.

2. The AB testing method in the field of travel dispatching according to claim 1, characterized in that: After the real-time calculation of the key business indicators of each synthetic control group in the process of executing the dispatching operation, the following is also included: During the future test phase, a preset multi-armed bandit algorithm is used to dynamically adjust the allocation of future time slices not performed by each synthetic control group based on each of the key business indicators.

3. The AB testing method in the field of travel dispatching according to claim 1, characterized in that: The step of selecting a target historical segment that meets a preset similar feature condition from a plurality of historical segments based on the historical order data corresponding to the historical period includes: Determine basic feature information corresponding to each of the historical segments based on the historical order data; wherein the basic feature information includes at least one of weather features, holiday features, and traffic conditions features; A target historical segment whose basic feature information matches the preset basic feature is selected from each of the historical segments.

4. The AB testing method in the field of travel dispatching according to claim 1, characterized in that: The step of selecting a target historical segment that meets a preset similar feature condition from a plurality of historical segments based on the historical order data corresponding to the historical period includes: Determine basic feature information and vehicle demand feature information corresponding to each of the historical segments based on the historical order data; wherein the basic feature information includes at least one of weather features, holiday features, and traffic conditions, and the vehicle demand feature information includes at least one of vehicle order distribution features, vehicle supply features, and vehicle distribution features; A candidate historical segment whose basic feature information matches the preset basic feature is screened out from each of the historical segments, and a target historical segment whose vehicle use demand feature information matches the preset vehicle use demand feature is screened out from each of the candidate historical segments.

5. The AB testing method in the field of travel dispatching according to claim 1, characterized in that: Also includes: Determine feature prediction information corresponding to each of the future time slices based on the historical order data; wherein the feature prediction information includes at least one of basic feature prediction information and vehicle demand feature prediction information, the basic feature prediction information includes at least one of weather features, holiday features, and traffic conditions, and the vehicle demand feature prediction information includes at least one of vehicle order distribution features, vehicle supply features, and vehicle distribution features; During the future test phase, actual feature monitoring information corresponding to each future time slice is acquired in real time; wherein the type of the actual feature monitoring information corresponds to the feature prediction information; When it is determined that the error between the feature prediction information and the actual feature monitoring information corresponding to the same future time slice exceeds a preset fault tolerance threshold, the experimental emergency adjustment strategy is executed; wherein, the experimental emergency adjustment strategy includes interrupting the experiment of the synthetic control group corresponding to this future time slice.

6. The AB testing method in the field of travel dispatching according to claim 1, characterized in that: The actual vehicle order flow generated in real time is divided according to the time and space attributes, and the corresponding actual vehicle order flow after the division is allocated to the corresponding synthetic control group, including: Based on the location information and time information of the actual car order generated in real time, the actual car order flow is mapped to a number of independent flow pools based on geographical regions according to a preset bucketing strategy, and at the same time, the actual car order flow in each independent flow pool is layered according to the unit time period; A target independent traffic pool matching the geographical area of ​​each of the synthetic control groups is determined, and the actual vehicle order traffic matching each of the future time slices in the target independent traffic pool is allocated to the corresponding synthetic control group.

7. An AB testing device in the field of travel dispatching, characterized in that: include: A time slice screening module, used to divide the historical period into a number of historical segments according to a preset unit time period, screen out target historical segments that meet preset similar feature conditions from the several historical segments based on the historical order data corresponding to the historical period, and determine a future time slice set corresponding to the future test stage based on each of the target historical segments; wherein the future time slices in the future time slice set correspond to each of the target historical segments one by one; A control group construction module, used for allocating future time slices in the future time slice set to obtain a plurality of synthetic control groups corresponding to the future test phase; A diversion and dispatching module is used to divert the actual vehicle order flow generated in real time according to the spatiotemporal attributes during the future test phase, and distribute the corresponding actual vehicle order flow after diversion to the corresponding synthetic control group, so that each synthetic control group performs the dispatching operation based on its own dispatching strategy; wherein the dispatching strategy includes at least two; The indicator calculation module is used to calculate the key business indicators of each synthetic control group in the process of executing the dispatching operation in real time during the future trial stage.

8. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the AB testing method in the field of travel dispatching as described in any one of claims 1-6 when executing the program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the AB testing method in the field of travel dispatching as described in any one of claims 1-6 is executed.

10. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the AB testing method in the field of travel dispatching described in any one of claims 1-6 is implemented.

Images