Data processing method, system, electronic device, storage medium and program product

Abstract

Embodiments of the present application provide a data processing method, system, electronic device, storage medium and program product. The method comprises: after receiving a test task sent for a target application, assigning a plurality of user accounts to a first number of first type data sets according to attribute information of the plurality of user accounts; a same user account is in one first type data set, and attribute distributions of user accounts contained in different first type data sets are consistent; generating a plurality of second type data sets based on the first number of first type data sets; at least two first type data sets are contained in each second type data set; and grouping the plurality of second type data sets to obtain data groups, and the data groups test a to-be-tested version of the target application. The present scheme can reduce errors of a system and random grouping, and can effectively ensure accuracy of a subsequent test result.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, system, electronic device, storage medium, and program product. Background Technology

[0002] With the development of technology, applications are being updated more and more frequently to meet user needs. Before updating an application, it is often necessary to conduct comparative testing and analysis of the old and new versions to determine whether the new version is superior to the old one before updating the application.

[0003] To complete testing, user data is typically first randomly grouped and assigned to different versions of the application to be analyzed (e.g., different pages, algorithm strategies, etc.), thus supporting subsequent data analysis for different versions of the application. However, existing application testing platforms mostly use hashing strategies to randomly group user data. Due to the limitations of hashing algorithms themselves and the uncertainty of whether a user has accessed the application, significant differences often exist between the groups, resulting in lower accuracy of subsequent test analysis results. Summary of the Invention

[0004] In view of the above problems, this application provides a data processing method, system, electronic device, storage medium, and program product that solves or at least partially solves the above problems.

[0005] In one embodiment of this application, a data processing method is provided. The method includes:

[0006] Upon receiving a test task for the target application, the user accounts are assigned to a first number of first-class datasets based on their attribute information; wherein, the same user account is in one first-class dataset, and the attribute distribution of user accounts in different first-class datasets is consistent;

[0007] Based on the first number of first-class datasets, multiple second-class datasets are generated; each second-class dataset contains at least two first-class datasets.

[0008] The plurality of second-class datasets are grouped to obtain data groups; the data groups are used to test the version of the target application.

[0009] Optionally, the above-described grouping of the plurality of second-type datasets to obtain data groups includes: determining a second number of groups; wherein the second number is a positive integer greater than 1; and grouping the plurality of second-type datasets according to the second number to obtain the second number of data groups.

[0010] Optionally, the above-described grouping of the plurality of second-type datasets according to the second quantity to obtain the second quantity of data groups includes: determining the first indicator data corresponding to each of the plurality of second-type datasets; based on the first indicator data, using a statistical grouping model to perform sampling group statistical analysis on the plurality of second-type datasets according to the second quantity, obtaining analysis results containing the data groups to which each second-type dataset belongs; when it is determined that the analysis results do not meet the termination condition of the sampling group statistical analysis, recording the analysis results, and returning to execute the step of performing sampling group statistical analysis on the plurality of second-type datasets according to the first indicator data and using a statistical grouping model according to the second quantity; when it is determined that the analysis results meet the termination condition, determining the second quantity of data groups based on the analysis results.

[0011] Optionally, satisfying the termination condition mentioned above includes: the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the difference requirement;

[0012] The second indicator data is determined based on the first indicator data corresponding to all second-class datasets included in the data group, and the first indicator data is determined based on the historical data of all user accounts included in the second-class dataset; the requirement to meet the difference includes at least one of the following: the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set threshold condition, and the weighted sum of the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set sum condition.

[0013] Optionally, the above-mentioned sampling and grouping of the plurality of second-type datasets according to the second quantity to obtain the second quantity of data groups includes: determining a plurality of grouped samples; wherein the data groups contained in the grouped samples are obtained by randomly grouping the plurality of second-type datasets according to the second quantity; when testing the test version of the target application using each grouped sample, using a prediction model to predict the second indicator data corresponding to each of the second quantity of data groups contained in each grouped sample, and obtaining prediction results; based on the prediction results, determining a target grouped sample from the plurality of grouped samples, and using the second quantity of data groups contained in the target grouped sample as the obtained second quantity of data groups.

[0014] Optionally, the above-mentioned prediction of the second indicator data corresponding to the second number of data groups contained in each group sample using the prediction model to obtain the prediction result includes: based on the attribute information of multiple user accounts, performing the prediction of the second indicator data corresponding to the second number of data groups contained in each group sample using the prediction model to obtain the prediction result.

[0015] Optionally, the aforementioned plurality of user accounts includes a plurality of first user accounts, wherein the first user accounts are user accounts whose attribute information meets preset conditions; and, the above-mentioned allocation of the plurality of user accounts to a first number of first-type datasets based on the attribute information of the plurality of user accounts includes: performing stratification processing on the plurality of first user accounts based on the attribute information of the plurality of first user accounts to obtain a plurality of sampling layers; wherein, the attribute distribution of different first user accounts on the same sampling layer is consistent; determining the sampling ratio corresponding to each sampling layer in the plurality of sampling layers; and performing stratified sampling on the first user accounts on the plurality of sampling layers according to the sampling ratio, so as to distribute the plurality of first user accounts to a first number of first-type datasets.

[0016] Optionally, the plurality of user accounts further includes a second user account, which is a user account whose attribute information does not meet the preset conditions; and, the above-mentioned allocation of the plurality of user accounts to a first number of first-type datasets based on the attribute information of the plurality of user accounts further includes: determining the dataset features corresponding to each first-type dataset based on the attribute information of the first user accounts contained in each of the first number of first-type datasets; and selecting, based on the dataset features, a first-type dataset matching the second user account from the first number of first-type datasets as the target first-type dataset allocated to the second user account.

[0017] In another embodiment of this application, a data processing apparatus is also provided. The apparatus includes:

[0018] The allocation module is used to receive a test task for a target application and, based on the attribute information of multiple user accounts, allocate the multiple user accounts to a first number of first-type datasets; wherein, the same user account is in one first-type dataset, and the attribute distribution of user accounts contained in different first-type datasets is consistent;

[0019] The generation module is used to generate multiple second-class datasets based on the first number of first-class datasets; the second-class datasets contain at least two first-class datasets;

[0020] The grouping module is used to group the multiple second-type datasets to obtain data groups; the data groups are used to test the version of the target application.

[0021] Optionally, the grouping module described above, when used to group the plurality of second-type datasets to obtain data groups, is specifically used to: determine a second number of groups; wherein the second number is a positive integer greater than 1; and group the plurality of second-type datasets according to the second number to obtain the second number of data groups.

[0022] Optionally, the grouping module described above, when used to group the plurality of second-type datasets according to the second quantity to obtain the second quantity of data groups, is specifically used for: determining the first indicator data corresponding to each of the plurality of second-type datasets; based on the first indicator data, using a statistical grouping model to perform sampling group statistical analysis on the plurality of second-type datasets according to the second quantity, obtaining analysis results containing the data groups to which each second-type dataset belongs; when it is determined that the analysis results do not meet the termination condition of the sampling group statistical analysis, recording the analysis results, and returning to execute the step of performing sampling group statistical analysis on the plurality of second-type datasets according to the first indicator data and using a statistical grouping model according to the second quantity; when it is determined that the analysis results meet the termination condition, determining the second quantity of data groups based on the analysis results.

[0023] Optionally, the analysis results described above satisfying the termination condition include: the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meet the difference requirement;

[0024] The second indicator data is determined based on the first indicator data corresponding to all second-class datasets included in the data group, and the first indicator data is determined based on the historical data of all user accounts included in the second-class dataset; the requirement to meet the difference includes at least one of the following: the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set threshold condition, and the weighted sum of the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set sum condition.

[0025] Optionally, the grouping module described above, when used to sample and group the plurality of second-type datasets according to the second quantity to obtain the second quantity of data groups, is specifically used for: determining a plurality of grouped samples; wherein the data groups contained in the grouped samples are obtained by randomly grouping the plurality of second-type datasets according to the second quantity; when using each grouped sample to test the version to be tested of the target application, using a prediction model to predict the second indicator data corresponding to the second quantity of data groups contained in each grouped sample, and obtaining prediction results; based on the prediction results, determining a target grouped sample from the plurality of grouped samples, and using the second quantity of data groups contained in the target grouped sample as the obtained second quantity of data groups.

[0026] Optionally, the aforementioned grouping module, when used to predict the second indicator data corresponding to the second number of data groups contained in each group sample using a prediction model, specifically uses the following methods to obtain prediction results:

[0027] Based on the attribute information of multiple user accounts, the prediction model is used to predict the second indicator data corresponding to the second number of data groups contained in each group sample, and the prediction results are obtained.

[0028] Optionally, the aforementioned plurality of user accounts includes a plurality of first user accounts, wherein the first user accounts are user accounts whose attribute information meets preset conditions; correspondingly, the aforementioned allocation module, when used to allocate the plurality of user accounts to a first number of first-type datasets according to the attribute information of the plurality of user accounts, is specifically used to: perform stratification processing on the plurality of first user accounts according to the attribute information of the plurality of first user accounts to obtain a plurality of sampling layers; wherein, the attribute distribution of different first user accounts on the same sampling layer is consistent; determine the sampling ratio corresponding to each sampling layer in the plurality of sampling layers; and perform stratified sampling on the first user accounts on the plurality of sampling layers according to the sampling ratio, so as to distribute the plurality of first user accounts to a first number of first-type datasets.

[0029] Optionally, the aforementioned plurality of user accounts further includes a second user account, which is a user account whose attribute information does not meet the preset conditions; correspondingly, the allocation module, when used to allocate the plurality of user accounts to a first number of first-type datasets based on the attribute information of the plurality of user accounts, specifically further includes: determining the dataset features corresponding to each first-type dataset based on the attribute information of the first user accounts contained in each of the first number of first-type datasets; and selecting at least one first-type dataset that matches the second user account from the first number of first-type datasets according to the dataset features, as the target first-type dataset allocated to the second user account.

[0030] In another embodiment of this application, a data processing system is also provided. The system includes:

[0031] The client is used to send test tasks for the target application to the server.

[0032] The server, upon receiving the test task, assigns multiple user accounts to a first number of first-type datasets based on the attribute information of multiple user accounts; wherein the same user account is within one first-type dataset, and the attribute distribution of user accounts contained in different first-type datasets is consistent; based on the first number of first-type datasets, multiple second-type datasets are generated; each second-type dataset contains at least two first-type datasets; the multiple second-type datasets are grouped to obtain data groups, which are used to test the version under test of the target application.

[0033] In another embodiment of this application, a data processing system is also provided. The system includes:

[0034] The application layer is used to provide test tasks for the target application to the test processing layer;

[0035] A test processing layer is used to, after obtaining the test task from the application layer, allocate multiple user accounts to a first number of first-type datasets based on the attribute information of multiple user accounts; wherein, the same user account is in one first-type dataset, and the attribute distribution of user accounts contained in different first-type datasets is consistent; based on the first number of first-type datasets, multiple second-type datasets are generated; each second-type dataset contains at least two first-type datasets; the multiple second-type datasets are grouped to obtain data groups; the data groups are used to test the version of the target application to be tested.

[0036] Optionally, the above data processing system further includes:

[0037] The data layer is used to interact with the database for data storage and retrieval;

[0038] The test processing layer is further configured to retrieve the plurality of user accounts from the database through the data layer; and to group and store the data in the database through the data layer.

[0039] In another embodiment of this application, an electronic device is also provided. The electronic device includes: a memory and a processor, wherein the memory is used to store one or more computer programs; the processor, coupled to the memory, is used to execute the one or more computer programs stored in the memory to implement the steps in the data processing method provided in the embodiments of this application.

[0040] In yet another embodiment of this application, a computer-readable medium is also provided. This computer-readable medium stores a computer program / instructions thereon, which, when executed by a processor, implement the steps of the data processing method provided in the embodiments of this application.

[0041] In yet another embodiment of this application, a computer program product is also provided. This computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the data processing method provided in the embodiments of this application.

[0042] The technical solution provided in this application, upon receiving a test task for a target application, allocates multiple user accounts to a first number of first-type datasets based on their attribute information. During this allocation process, the same user account resides within one first-type dataset, and the attribute distribution of user accounts in different first-type datasets is consistent. This effectively ensures the representativeness of all user accounts in each first-type dataset to the overall user account count, thus reducing system errors. Furthermore, based on the first number of first-type datasets, multiple second-type datasets containing at least two first-type datasets are generated. These second-type datasets are then grouped to obtain corresponding data groups. This grouping method reduces the error of random grouping and effectively ensures minimal differences between different data groups. This allows for more realistic and accurate test results when subsequently testing the target application's version using these data groups, avoiding redundant testing and improving testing efficiency. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1a This diagram illustrates the principle of using a hashing strategy to break down and group multiple user accounts into several datasets in an existing scheme.

[0045] Figure 1b This diagram illustrates the principle of grouping multiple datasets contained in the test layer in an existing scheme.

[0046] Figure 2 A schematic flowchart of a data processing method provided in an embodiment of this application is shown;

[0047] Figure 3 This application is shown to be aimed at Figure 1a and Figure 1b The diagram shows a schematic representation of the principle of the proposed optimization scheme from the existing solution.

[0048] Figure 4 This illustration shows a schematic diagram illustrating the principle of grouping multiple second-type datasets according to an embodiment of this application;

[0049] Figure 5a A schematic diagram of the structure of a data processing system provided in an embodiment of this application is shown;

[0050] Figure 5bThis application shows a schematic diagram illustrating the specific form of a client and server provided in one embodiment.

[0051] Figure 6 A schematic diagram of the structure of a data processing system provided in another embodiment of this application is shown;

[0052] Figure 7 A schematic diagram of the structure of a data processing apparatus provided in an embodiment of this application is shown;

[0053] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0054] Before introducing the technical solutions provided in this application, a brief explanation of the technical terms involved in this application will be given.

[0055] A testing platform is used to test different versions of a test subject. Specifically, for at least two versions defined for the test subject, the testing platform randomly divides different user accounts into at least two groups (one group serves as the control group, and the other at least one group serves as the test group) that maintain the same (or similar) attribute characteristic distribution. Then, different versions are run on at least two groups of user accounts, and feedback data from each group of user accounts is collected, analyzed, and evaluated to determine the corresponding indicator data for each version. Finally, based on the indicator data, each version can be evaluated, and the version with the best performance can be selected as the final adopted version.

[0056] The aforementioned test object can be a product. For example, before updating an application, the application can be tested to determine the effect of the new version on users compared to the old version. The old and new versions of the application are then two versions specifically designed for the application product. Alternatively, the test object can be one or more elements within a product. For instance, testing the button colors in an application involves using button colors as the test object, with red, blue, and yellow buttons serving as three versions for testing. It should be noted that the embodiments described below are illustrated using an application as an example.

[0057] Metric data, which includes, but is not limited to, metrics such as clicks, page view duration, and purchases. Metrics can be determined based on the test subject; different test subjects may use different metrics. For example, when testing webpage background color, and using versions with white, yellow, and light green backgrounds, the metric in this test could be page view duration. If the light green background version has the longest page view duration during the test, it indicates that the light green background version is performing better. In a test, when evaluating the versions participating in the test, a single metric or multiple metrics can be used. When using multiple metrics, the effectiveness of each version can be determined by combining the results of all metrics.

[0058] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0059] All information collected in this application is fully authorized by all parties and is used solely for testing purposes.

[0060] In some processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. These operations may be executed out of order or in parallel. Operation numbers such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the terms "first," "second," etc., used herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types. The term "or / and" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A or / and B means that A can exist alone, A and B can exist simultaneously, or B can exist alone. The character " / " in this application generally indicates that the preceding and following related objects have an "or" relationship. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system including said element. Furthermore, the following embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0061] In practical applications, before iteratively updating an application, it is often necessary to test both the old and new versions. This testing is primarily accomplished using a testing platform (such as an A / B testing platform). Since there are often many tests running concurrently, to ensure that multiple tests can run in parallel without interfering with each other, testing platforms typically employ a multi-layered online testing mode (e.g., ...). Figure 1a As shown in the diagram, a test layer refers to a collection of a type of test (such as UI tests, algorithm tests, etc.), or in other words, a test layer is responsible for managing a type of test task. The user account grouping methods for each test layer are completely independent and do not affect each other. When the testing platform uses horizontal test layers, each test layer possesses all user accounts (i.e., 100% user accounts), and tests within the same test layer share 100% of the user accounts for that test layer. When these 100% user accounts traverse each test layer, they are randomly regrouped again, and the randomization effect is discrete. The 100% user accounts possessed by each test layer are the same batch of user accounts; in other words, user accounts between different test layers are reused. For example, see [link to relevant documentation]. Figure 1a Test layer 2 reuses 100% of the user accounts from test layer 1.

[0062] One relevant technical solution involves using a hash strategy to distribute user accounts, specifically: see [link to relevant documentation]. Figure 1a As shown, different hash factors (denoted as Seed) are used for different test layers to randomly distribute 100% of user accounts into a predetermined number of datasets for the corresponding test layer. For example, let D represent the dataset, Di... j This indicates that the dataset is the j-th dataset in test layer i. Taking test layer 1 as an example, assume that test layer 1 contains 1000 datasets of the first dataset (i.e., Figure 1a D11~D1 shown in the figure 1000 Given that the first hash factor corresponding to Test Layer 1 is Seed1, and userID represents the identifier corresponding to a user account, when allocating the corresponding first dataset to a user account among 100% of the user accounts, the identifier userID corresponding to that user account can be concatenated with Seed1 to obtain a string. This string is then subjected to MD5 (Message Digest Algorithm MD5, also known as a hash algorithm, is a type of hash function) operation to obtain a fixed-length hash value. Then, using a preset number (i.e., 1000) of the first dataset in Test Layer 1, the hash value is modulo-calculated. The calculated result is the identifier of the first dataset allocated to that user account. In this way, 100% of the user accounts can be allocated to the 1000 first datasets contained in Test Layer 1. Further, see... Figure 1b As shown, assuming the target test layer is test layer 1, the 1000 first datasets on test layer 1 will be randomly grouped according to the test tasks created by the client user (or test user) on test layer 1. These 1000 first datasets will be divided into at least two data groups that are adapted to the test tasks, so that the at least two data groups can be used to test and analyze at least two test versions of the test object (such as the target application) to evaluate the performance of the at least two test versions. Figure 1b The diagram illustrates dividing the 1000 datasets on test layer 1 into two data groups (i.e., Figure 1b The examples shown are Group 1 and Group 2, where the datasets contained in these two groups are identified using different grayscale colors. Additionally, for easier distinction, in... Figure 1bThe process described above, which uses a hashing strategy to distribute 100% of user accounts across 1000 first datasets in test layer 1, is referred to as the first randomization (Random1). The process of randomly grouping the 1000 first datasets in test layer 1 into two groups is referred to as the second randomization (Random2).

[0063] It should be noted that the datasets contained in each of the above test layers can also be called buckets. Accordingly, allocating 100% of user accounts to a preset number of datasets is equivalent to allocating 100% of user accounts to a preset number of buckets.

[0064] for Figure 1a and Figure 1b The existing scheme shown in the diagram, in its method of grouping user accounts, during the initial random allocation of 100% of user accounts to a predetermined number of datasets in the corresponding test layer using a hash strategy, suffers from inherent limitations in the hash algorithm itself. This can lead to uneven distribution of the number and attributes of user accounts, potentially causing inter-layer interference. For example, see [link to previous section]. Figure 1a As shown, 100% of the user accounts in test layer 2 are reused from user accounts in test layer 1. It is assumed that test layer 2 contains 1000 second datasets (i.e., ...). Figure 1a D21 to D2 shown in the figure 1000 The second hash factor for test layer 2 is Seed2. During the reuse of 100% of the user accounts from test layer 1 in test layer 2, the second hash factor (Seed2) will be used to redistribute the user accounts from each first dataset in test layer 1 to the 1000 second datasets in test layer 2. Due to the uneven distribution of user accounts and attribute characteristics among the different first datasets in test layer 1, this inevitably affects the performance of the 1000 second datasets in test layer 2, i.e., inter-layer interference. Furthermore, during the second random grouping of a preset number of datasets from the test layer (e.g., test layer 1), due to the influence of past tests and user visits, there will be inconsistencies in the distribution of user attribute characteristics within each data group; these user attribute characteristics may include, but are not limited to, inherent attributes (such as education level and age) and behavioral attributes (such as activity level and application usage). The inter-layer interference and biases in user attribute distribution caused by the above factors appear to testers using testing platforms as AA-diff (significant differences in metrics before the product / strategy takes effect). This not only prolongs the testing cycle and amplifies Type I and Type II error rates, but may also lead to unpredictable or even erroneous estimates of the product / strategy's effectiveness. Type I error refers to the probability of rejecting the null hypothesis when it is true; Type II error refers to the probability of accepting the null hypothesis when it is false.

[0065] To address the aforementioned AA-diff problem, this application provides a data processing solution. In this application's solution, the existing methods that utilize methods such as... Figure 1b The two randomizations shown (Random1 and Random2) are used to group user accounts, each optimized with a different sampling scheme. By controlling the sampled user accounts, the impact of AA-diff is minimized, and the statistical inference method is fundamentally assisted in “correcting” the impact of AA-diff on AB-diff (the observed product / strategy effect during the test), thereby obtaining a true and accurate estimate of the test.

[0066] When optimizing the two randomizations in the existing scheme, considering that the poor randomization results in the existing scheme correspond to systematic error and random error respectively from the perspective of error, the scheme of this application will optimize the existing scheme based on the idea of ​​unbiasedness, so that the user accounts contained in the data group used to test the test object (such as the target application) can not only meet the representativeness of the overall user accounts, but also ensure the unbiasedness between the data groups. In other words, it ensures that the user accounts contained in the data groups have consistent performance, such as the user accounts contained in the data groups having the same or similar attribute feature distribution.

[0067] The technical solution provided in this application will be described in detail below.

[0068] Figure 2 A flowchart illustrating a data processing method according to an embodiment of this application is shown. The execution entity of this method can be a server, which can be, but is not limited to, a single server, a server cluster consisting of multiple single servers, a virtual server, a cloud server, etc., and is not limited here. A testing platform is integrated on the server; correspondingly, the specific execution entity of the method provided in this embodiment is the testing platform on the server. The testing platform can be used to initiate tests and specify test plans for received test tasks, such as the test versions included in the tests, grouping multiple user accounts to allocate user account data to each test version, test lifecycle, test metrics, etc. Furthermore, the testing platform can also determine user account grouping strategies for the tests. For a detailed introduction to the testing platform, please refer to the following description of the data processing system; it will not be elaborated upon here. Figure 2 The data processing method provided in this embodiment includes the following steps:

[0069] 101. Upon receiving a test task for the target application, the user accounts are assigned to a first number of first-class datasets based on their attribute information; wherein, the same user is in one first-class dataset, and the attribute distribution of the sewing user accounts contained in different first-class datasets is consistent.

[0070] 102. Based on the first number of first-class datasets, generate multiple second-class datasets; each second-class dataset contains at least two first-class datasets;

[0071] 103. Group the plurality of second-class datasets to obtain data groups; the data groups are used to test the version of the target application to be tested.

[0072] In the above 101, the target application can be an application configured on a terminal device, such as a video application, instant messaging application, browser application, educational application, live streaming application, etc. This embodiment does not limit the type of target application. The terminal device can be, but is not limited to, mobile phones, laptops, tablets, desktop computers, smart TVs, smart wearable devices (such as smartwatches), etc. At least two test versions can be created for the target application for testing; for details on the versions, please refer to the relevant content above.

[0073] In practical applications, the testing platform provides an interactive interface for clients to access, allowing them to configure relevant test tasks for the target application. Test tasks may include, but are not limited to, basic test information, version information of the version under test, related metrics, and target audience. Basic test information may include, but is not limited to, test name and test description. Version information may include, but is not limited to, push channel (e.g., Getui channel), push platform (Android, iOS, etc.), push title, and push content. Related metrics refer to the metrics associated with each test used to evaluate the merits of different versions. Generally, the default metric associated with a test is the North Star metric (also called the first critical metric). The North Star metric is the absolute core metric related to the business / strategy during the product's initial signing phase. It can be used to clearly indicate the functionalities the product wants to convey and the directions that need optimization at each product stage, guide the collaborative work of staff in different product teams, etc. Of course, client users can also associate other types of metrics for tests, such as DAU (Daily Active Users), daily new users, retention rate, activity rate, conversion rate, visit duration, etc., which are not limited here. The target audience, or target users, refers to the users selected to participate in testing the target application (also called the test population). Test users can be divided into full users and targeted users. Full users refer to all users without any restrictions. Specifically, full users can refer to all users in the user data pool (i.e., 100% of users). Full users include existing users and new users. Existing users refer to users who already exist in the user data pool, or historical users, while new users refer to users newly added to the user pool. Targeted users refer to specific categories of users identified based on user profiles and behaviors, such as users in a particular city or users with preferences for movie channels. Based on the above, the test tasks obtained in 101 above can be sent by the client. The client can obtain test tasks for the target application by responding to configuration operations triggered by the client user for the target application; the configuration operations can be completed on the interactive interface provided by the test platform called by the client, and the client can be, but is not limited to, smartphones, tablets, computers, etc.

[0074] Furthermore, upon receiving a test task, multiple user accounts matching the specified user class can be selected from a user data pool (hereinafter referred to as the data pool) that stores user accounts, based on the user class specified in the test task. These users will then serve as target users for subsequent testing of the target application. For example, assuming the specified user class in the test task is all users, the user accounts matching the specified user class will be all user accounts in the data pool. In other words, all users in the data pool will serve as target users for subsequent testing of the target application.

[0075] After selecting multiple user accounts, in order to reduce system errors, unlike... Figure 1a The existing solution shown directly assigns multiple user accounts to multiple datasets in different test layers based on the hash factors of different test layers. The technical solution provided in this embodiment determines a first number of first-class datasets based on the attribute information of multiple user accounts. In specific implementation, a stratified sampling strategy is first used to distribute multiple user accounts to a first number of first-class datasets (e.g., ...). Figure 3 or Figure 5a (As shown). More specifically, based on the attribute information of multiple user accounts, cross-stratified sampling is performed on these user accounts to distribute them across a first number of first-class datasets. The first number of first-class datasets can be a default value, such as 2000, or can be flexibly set by the client user according to the testing tasks of the target application; this is not limited here. In the above allocation process, the same user account is within one first-class dataset, and the attribute distribution of user accounts in different first-class datasets is consistent. For example, different first-class datasets may contain the same number of user accounts, or the user attribute feature distributions corresponding to user accounts in different first-class datasets may be the same or similar.

[0076] In practical implementation, when performing cross-stratified sampling on multiple user accounts, it is considered that these user accounts may include two types: existing user accounts (also called historical user accounts) and incremental user accounts (also called new user accounts). Historical user accounts refer to user accounts that existed in the past within a certain period; in simple terms, they are old user accounts, as opposed to new user accounts. Compared to new user accounts, historical user accounts are relatively more numerous and contain richer user attribute information. For example, the user attribute information of new user accounts is generally only inherent attributes such as age and education level, while the user attribute characteristics of historical user accounts include not only inherent attributes but also behavioral attributes such as user activity and application usage. However, considering that historical user accounts may have poor activity levels, resulting in attribute information containing only inherent attributes, or containing only a negligible amount of behavioral attributes along with inherent attributes, this embodiment refers to user accounts whose attribute information meets preset conditions as first user accounts, and user accounts whose attribute information does not meet preset conditions as second user accounts. For example, user accounts whose creation time is longer than the preset time can be identified as the first user account, while user accounts whose creation time is less than or equal to the preset time can be identified as the second user account.

[0077] To ensure consistency in user account performance across different first-class datasets, cross-stratified sampling is used to assign first-class user accounts to their respective first-class datasets. For second-class user accounts, the first-class dataset to be assigned is determined by combining the available attribute information of the second-class user accounts with the dataset features corresponding to each first-class dataset.

[0078] That is, in one feasible technical solution, the plurality of user accounts includes a plurality of first user accounts, wherein the first user accounts are user accounts whose attribute information meets preset conditions. The preset conditions for attribute information to meet preset conditions include at least one of the following: a time attribute value greater than or equal to a preset duration, or an activity attribute value greater than or equal to a preset activity value. Specifically, a time attribute value greater than or equal to a preset duration may include, but is not limited to: creation time greater than or equal to a preset duration, online time (such as average daily online time or cumulative total online time) greater than or equal to a preset duration, etc.; an activity attribute value greater than or equal to a preset activity value may include, but is not limited to: login count greater than or equal to a preset number of times, number of likes, comments, collections, etc. greater than or equal to a preset number of times, login interval less than or equal to a preset time, etc.

[0079] Furthermore, the phrase "assigning multiple user accounts to a first number of first-class datasets based on the attribute information of multiple user accounts" in section 101 above can specifically include:

[0080] 1011. Based on the attribute information of each first user account, the multiple first user accounts are stratified to obtain multiple sampling layers; wherein, the attribute information corresponding to different first user accounts in the same sampling layer is consistent.

[0081] 1012. Determine the sampling ratio corresponding to each sampling layer in the plurality of sampling layers;

[0082] 1013. According to the sampling ratio, perform stratified sampling on the first user accounts in the plurality of sampling layers to distribute the plurality of first user accounts into a first number of first type datasets.

[0083] In steps 1010 and 1011 above, the attribute information of the user account may include at least one of the following: the user's inherent attributes and the user's behavioral attributes; wherein, the inherent attributes may be gender, age, education level, etc., and the behavioral attributes may be activity level, application usage, etc. Based on the attribute information of each first user account, the overall first user accounts can be cross-stratified according to multiple attributes and the number of segments corresponding to each attribute, thereby obtaining multiple sampling layers.

[0084] Specifically, the principle of cross-layer processing can be understood as follows:

[0085] First, determine the various attributes and the number of segments for each attribute. For example, when the target application is a video application, we can determine two attributes: gender and activity level. Activity level can refer to the user's daily online time, which can be, but is not limited to, the average daily online time obtained through statistical analysis of the user's historical data. For gender and activity level, we can determine that the number of segments for gender is 2, divided into male and female, and the number of segments for activity level is 3, divided into less than 3 hours (inclusive), 3 to 8 hours (exclusive), and more than 8 hours (inclusive).

[0086] Then, based on the attribute information of each first user account, the multiple first user accounts can be segmented according to gender to obtain two major segments: male and female. Further, the first user accounts contained in each segment of the male and female segments can be segmented according to activity level to obtain three sub-segments corresponding to each of the male and female segments. The resulting six sub-segments are the final multiple sampling layers.

[0087] Based on the principles of cross-stratification described above, it can be understood that if there are N determined attributes, the number of segments for the first attribute is A1, the number of segments for the second attribute is A2, and so on until the number of segments for the Nth attribute is An, then the total number of sampling layers obtained is A1*A2*......*An.

[0088] It should be noted that during cross-segmentation, it is ensured that a first user account is on a sampling layer, and different first user accounts on the same sampling layer have the same or similar attribute characteristics.

[0089] In step 1012 above, the sampling ratio corresponding to each sampling layer in multiple sampling layers can be determined based on the number of first user accounts included in each sampling layer and the total number of first user accounts. Further, in step 1013 above, based on the number of first user accounts to be allocated to each first type of dataset determined according to the total number of first user accounts and the first number of first type datasets, the extraction quantity of first user accounts corresponding to each sampling layer can be determined based on the sampling ratio and the number of first user accounts to be allocated to each first type dataset. Then, stratified sampling is performed according to the extraction quantity of first user accounts corresponding to each sampling layer to allocate multiple first user data sets to a first number of first type datasets. In the above, an auto-incrementing numbering method can be used to number the first number of first type datasets. During the stratified sampling process, a correspondence between the number of stratified samplings and the numbering of the first type datasets is established, thereby allocating the first user accounts extracted from each sampling layer in the current stratified sampling to the corresponding first type dataset.

[0090] Regarding the content described in 1012-1013 above, combined with Figure 3 To give a specific example, suppose Figure 3 The diagram shows that the total number of first user accounts contained in 100% of the user accounts is 200,000. These 200,000 users are divided into three sampling layers: sampling layer 1, sampling layer 2, and sampling layer 3. Let the proportion of the number of first user accounts contained in sampling layer 1, sampling layer 2, and sampling layer 3 to the total number of first user accounts be 2 / 5, 3 / 10, and 3 / 10, respectively. In other words, the sampling ratios for sampling layers 1, 2, and 3 are 2 / 5, 3 / 10, and 3 / 10, respectively. Further, let 'd' represent the first type of dataset, and let the first number of the first type of dataset be 2000. These 2000 first type datasets (i.e.,...) Figure 3 The d1 to d2 shown in the figure 2000 The numbers of the samples are 1, 2, ..., m-1, m..., 1999, 2000. Therefore, the number of first user accounts allocated to each first type of dataset should be 100. To this end, a stratified sampling operation is performed on sampling layers 1, 2, and 3. The number of first user accounts extracted from sampling layers 1, 2, and 3 are 40, 30, and 30, respectively. Based on the determined number of first user accounts extracted from each sampling layer, the stratified sampling operation is performed on sampling layers 1, 2, and 3. During stratified sampling, the 100 first-user accounts sampled from sampling layers 1, 2, and 3 in the first stratified sampling operation are assigned to the first-class dataset d1 (numbered 1) according to the established correspondence between the number of stratified sampling operations and the numbering of the first-class datasets. Similarly, the 100 first-user accounts sampled from sampling layers 1, 2, and 3 in the second stratified sampling operation are assigned to the first-class dataset d2 (numbered 2), and so on, until all 200,000 first-user accounts are distributed among 2,000 first-class datasets. It should be noted that sampling layers 1, 2, and 3 are independent of each other. When sampling from sampling layers 1, 2, and 3 respectively, the sampling methods used for each layer can be the same or different; this is not limited here. For example, random sampling can be used for sampling layers 1, 2, and 3; or, random sampling can be used for sampling layer 1, and hash sampling can be used for sampling layers 2 and 3.

[0091] Because stratified sampling is used for the first user account, it can effectively ensure that each first user account sampled in the stratified sampling operation can be consistent with the overall first user account in every attribute dimension. This can effectively ensure the consistency between the first type of datasets and facilitate the determination of more accurate data groups in the future.

[0092] Furthermore, in another feasible technical solution, the multiple user accounts also include a second user account; the second user account is a user account whose attribute information does not meet the preset conditions, wherein the attribute information not meeting the preset conditions includes at least one of the following: the time attribute value is less than a preset duration, and the activity attribute value is less than a preset activity value. Specifically, the time attribute value being less than the preset duration may include, but is not limited to: creation time being less than the preset duration, online time (such as average daily online time or cumulative total online time) being less than the preset duration, etc.; the activity attribute value being less than the preset activity value may include, but is not limited to: login count being less than the preset count, number of likes, comments, collections, etc. being less than the preset count, login interval time being less than the preset time, etc.

[0093] Furthermore, the step 101 above, "assigning multiple user accounts to a first number of first-type datasets based on the attribute information of multiple user accounts," may further include the following steps:

[0094] 1014. Based on the attribute information of the first user accounts contained in each of the first number of first type datasets, determine the dataset features corresponding to each first type dataset;

[0095] 1015. Based on the characteristics of the dataset, select a first type of dataset that matches the second user account from the first number of first type datasets, and use it as the target first type dataset to be assigned to the second user account.

[0096] In practice, the dataset features corresponding to each first-class dataset can be calculated by weighted averaging of the attribute information of all user accounts included in that first-class dataset. When selecting a matching first-class dataset for a second user account, based on the dataset features corresponding to each first-class dataset, the total attribute information of multiple user accounts and the available attribute information (such as inherent attributes) of the second user account can be further combined to select a first-class dataset that matches the second user account from a first number of first-class datasets, which will be used as the target first-class dataset assigned to the second user account. The number of second user accounts can be one or more (i.e., two or more), and correspondingly, the number of first-class datasets selected that match the second user account can also be one or more, without limitation here.

[0097] For example, suppose that the age of Zhang San, a second user account, is 30, and the average age of the multiple user accounts is 20. The average age of all user accounts in the first dataset numbered 200 is 18 (less than 20). Therefore, Zhang San can be assigned to the first dataset numbered 200. By combining the available attribute information of the second user account with the characteristics of each first dataset, this method of assigning the second user account to the corresponding first dataset can further effectively ensure the consistency of each first dataset. This ensures that the user accounts contained in each first dataset can represent the overall user accounts, thus facilitating better data grouping in the future.

[0098] It should be noted that the consistency of the above-mentioned first-class datasets means that the number of user accounts corresponding to the user accounts included in each first-class dataset is equal, and the distribution of the corresponding attribute information is the same or similar, etc. In other words, the user accounts included in each first-class dataset have consistent performance.

[0099] Furthermore, after distributing multiple user accounts across a first number of first-class datasets, second-class datasets can be generated based on these first number of first-class datasets. One of the pre-built test layers is then designated as the target test layer. Subsequently, data grouping is achieved by grouping the multiple second-class datasets contained within the target test layer. That is, in one example, the aforementioned 102 "generating multiple second-class datasets based on a first number of first-class datasets" can specifically include:

[0100] A102. Based on a first number of first-class datasets, generate multiple second-class datasets contained in the target test layer;

[0101] Furthermore, a specific achievable technical solution for the above-mentioned A102 "generating multiple second-class datasets contained in the target test layer based on a first number of first-class datasets" is as follows:

[0102] 1021. Obtain multiple pre-built test layers;

[0103] 1022. Based on the first number of first-class datasets, generate multiple second-class datasets contained in each of the multiple test layers;

[0104] 1023. Select one test layer from the plurality of test layers as the target test layer.

[0105] The reason why it is necessary to generate other second-class datasets contained in the test layer, excluding the second-class datasets contained in the target test layer, based on a first number of first-class datasets will be explained in the following text and will not be repeated here.

[0106] In sections 1021 and 1022 above, multiple test layers are pre-built for the target application using a testing platform. A test layer refers to a collection of a type of test (such as UI tests, algorithm tests, etc.). The configuration information of each test layer may contain the number of second-type datasets included in that test layer. Based on a first number of first-type datasets, multiple second-type datasets included in each of the multiple test layers can be generated according to the number of second-type datasets that can be included in each test layer. In the process of generating multiple second-type datasets included in a test layer, a random equivalence method or a hash algorithm may be used, but is not limited to, to group the first number of first-type datasets to generate multiple second-type datasets included in the test layer. The number of second-type datasets included in each configured test layer is less than the number of first-type datasets, and each second-type dataset contains at least two first-type datasets.

[0107] For example, consider hash algorithms; see further. Figure 3 Continuing from the above example, assuming the first number of the first type of dataset is 2000, and the number of the second type of datasets included in each test layer is set to 1000, in the process of generating 1000 second type of datasets included in test layer 1 based on the 2000 first type of datasets, the number of the first type of dataset can be concatenated with the hash factor corresponding to test layer 1 to obtain a string. The string can then be subjected to MD5 operation to obtain a hash value of fixed length. Then, the hash value can be moduloed using the number of second type of datasets included in test layer 1. The result of the calculation identifies the first type of dataset in relation to the assigned second type of dataset. In this way, multiple second type of datasets included in test layer 1 can be generated based on the first number of first type of datasets.

[0108] In addition to including the number of second-type datasets configured for the test layer, the configuration information of the aforementioned test layer may also include adaptation information configured for the test layer. The adaptation information may include the type of the test task being adapted; only test tasks that meet the requirements of the corresponding test task type will use this test layer. Based on this, in one embodiment, in step 1023 above, a test layer matching the task type of the test task can be selected from multiple test layers as the target test layer, based on the task type of the test task. Furthermore, the adaptation information may also include at least one of the following: client name, adapted platform name, and adapted client version number. Only clients that meet the adaptation information can access the test layer. Therefore, in step 1023 above, a test layer that is adapted to the client and matches the task type of the test task can also be selected from multiple test layers based on client information as the target test layer. Client information may include, but is not limited to, client name, platform name, and client version number; where the client name is used to distinguish different clients, the platform name refers to the operating system on which the client relies, such as Android, iOS, Symbian, and Microsoft, etc., and the client version number indicates which version the client is currently running.

[0109] Furthermore, after identifying the multiple second-type datasets included in the target test layer, these datasets can be grouped according to a certain number of groupings to obtain a corresponding number of data groups. That is, in a specific implementable technical solution, step 103 above, "grouping the multiple second-type datasets to obtain data groups," can specifically include the following steps:

[0110] 1031. Determine the second number of groups; where the second number is a positive integer greater than 1;

[0111] 1032. Group the plurality of second-class datasets according to the second quantity to obtain the second quantity of data groups;

[0112] One data group is used to test a version of the target application.

[0113] In the above 1031, the second number of groups can be determined based on the test version information included in the test task. For example, if the client user creates two test versions for the target application (such as an old version and a new version), the second number of groups is 2; if the client user creates three test versions for the target application (such as an old version and two new versions), the second number of groups is 3. This embodiment does not specifically limit the second number of groups, but the second number of groups is at least two. The test platform can default to setting the second number of groups to 2, and the client user can flexibly modify it according to the actual situation.

[0114] In the aforementioned 1032, considering that after obtaining the test task for the target application, multiple user accounts are first cross-stratified sampled and then distributed into a first number of first-class datasets, although this can better ensure the consistency of user account performance in each of the first number of first-class datasets from a macro perspective, and can ensure the representativeness of all user accounts in each first-class dataset to the overall user accounts, thus reducing inter-layer interference when generating multiple second-class datasets for each test layer based on the first number of first-class datasets, due to the uncontrollable factor of whether users access the target application, to maintain good homogeneity (i.e., consistency) between different groups when grouping multiple second-class datasets of the target test layer, a more precise grouping method than the random grouping in existing technical solutions is required. To address this problem, this embodiment provides a grouping method based on a combination of statistics and iterative sampling, which is implemented through an established statistical grouping model.

[0115] In practical implementation, the aforementioned statistical grouping model is a reinforcement learning-based model. Specifically, this statistical grouping model can be, but is not limited to, constructed based on the Monte Carlo (MC) method in reinforcement learning. The MC method, also called a statistical simulation method or a stochastic simulation method, is a numerical method based on probability theory and mathematical statistics. Its principle is to conduct random simulation tests on a computer to solve a probability model related to a certain problem, thereby approximating the result of the problem. Specifically, the MC method is based on statistical sampling theory, using random numbers to conduct sampling tests or random simulations on existing data of random variables to obtain a certain numerical characteristic of the statistic and use it as the numerical solution to the problem. Based on this, in one feasible technical solution, the aforementioned 1032 "grouping the multiple second-class datasets according to the second quantity to obtain the second quantity of data groups" can be implemented using the following steps:

[0116] 10320. Determine the first indicator data corresponding to each of the plurality of second-class datasets;

[0117] 10321. Based on the first indicator data, a statistical grouping model is used to perform sampling grouping statistical analysis on the multiple second-class datasets according to the second quantity, so as to obtain the analysis results containing the data groups to which each second-class dataset belongs;

[0118] 10322. Determine whether the termination conditions for the sampling group statistical analysis are met;

[0119] 10323. If the termination condition is not met, record the analysis results and return to the step of performing the sampling group statistical analysis of the multiple second-class datasets based on the first indicator data and using the statistical grouping model according to the second quantity;

[0120] 10324. When the termination condition is met, determine the second number of data groups based on the analysis results.

[0121] In the above 10320, the indicator data, such as the first indicator data in this step or the second indicator data described below, includes at least one indicator that refers to the associated indicator set by the client user for the target application when setting up a test task for the target application. Specifically, the associated indicator can be, but is not limited to, the North Star indicator, DAU, retention rate, access duration, etc. For a detailed introduction to the associated indicator, please refer to the relevant content above, which will not be elaborated here.

[0122] Specifically, when determining the first indicator data corresponding to a second-type dataset, the indicator data corresponding to a first number of first-type datasets can be calculated based on several days of historical data corresponding to multiple user accounts. Then, a weighted sum of the indicator data corresponding to all first-type datasets included in the second-type dataset is performed to obtain the indicator data corresponding to the second-type dataset. Specifically, when calculating the indicator data corresponding to a first-type dataset, the daily indicator data for each of the first-type dataset over those several days can be calculated based on several days of historical data corresponding to all user accounts included in the first-type dataset. Then, the daily indicator data for each of the first-type dataset over those several days is normalized on a daily basis to finally obtain the indicator data corresponding to the first-type dataset.

[0123] In the above 10321, multiple second-class datasets and the first indicator data corresponding to each second-class dataset can be used as input to the statistical grouping model. Then, by executing the statistical grouping model, the statistical grouping model performs a simulated statistical grouping test on multiple second-class datasets based on the first indicator data corresponding to each second-class dataset, thereby obtaining the corresponding analysis results containing the data groups to which each second-class dataset belongs.

[0124] Among the above 10332, the termination criteria for sampling group statistical analysis may include at least one of the following:

[0125] S11. The number of times the sampling group statistical analysis is performed reaches the set number;

[0126] S12. The differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meet the difference requirements;

[0127] The second indicator data is determined based on the first indicator data corresponding to all second-class datasets included in the data group, and the first indicator data is determined based on the historical data of all user accounts included in the second-class dataset; the requirement to meet the difference includes at least one of the following: the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set threshold condition, and the weighted sum of the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set sum condition.

[0128] In S11 above, the number of times can be indirectly calculated based on the total working time preset for the statistical grouping model, or the number of times can be set directly, such as 10 times, 15 times, 20 times, etc.

[0129] In S12 above, the second indicator data corresponding to a data group can be obtained, but is not limited to, by calculating the weighted sum of the first indicator data corresponding to each of the second type of datasets contained in the data group.

[0130] In one embodiment, when the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups meets a set threshold condition, the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups is considered to meet the difference requirement. In some embodiments, the set threshold can be a fixed value; that is, the set threshold can be a constant value that can be flexibly set according to actual conditions, and different thresholds can be set for different indicator data. In this case, if the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups (for ease of description, this difference is denoted as the first difference) is greater than the set threshold, it indicates that there is a significant difference between the second number of data groups, and under this condition, the sampling group statistical analysis can be determined to not meet the termination condition. Conversely, if the first difference is less than or equal to the set threshold (i.e., the first difference meets the set threshold), it indicates that the difference between the second number of data groups is small and can be ignored, and under this condition, the sampling group statistical analysis can be determined to meet the termination condition.

[0131] In some other embodiments, the aforementioned threshold can also be a range value. If the first difference exceeds this range value, the sampling group statistical analysis is determined not to meet the termination condition. Only when the first difference is within the range value is the first difference considered to meet the threshold value, and the sampling group statistical analysis is deemed to meet the termination condition. The aforementioned range value is determined by the first indicator data corresponding to each second type of dataset in multiple test layers other than the target test layer, calculated based on several days of historical data corresponding to multiple user accounts. Specifically, the range value determination process can be as follows:

[0132] Assuming the target test layer is test layer 1, for other test layers (such as test layer 2 and test layer 3), based on several days of historical data corresponding to multiple user accounts, methods such as bootstrap can be used to calculate offline the fluctuation range of different evaluation indicators for each day within several historical records at the minimum sampling granularity (or commonly used sampling granularity). The minimum sampling granularity is related to the number of second-class datasets contained in the test layer. For example, if the test layer is a thousand-level layer, meaning it contains 1000 second-class datasets, then the minimum sampling granularity is one-thousandth. Therefore, in the above analysis of the fluctuation range of different evaluation indicators for each day, the analysis essentially uses the second-class dataset as a unit, calculating the indicator data corresponding to each second-class dataset contained in the test layer for each day within several historical records based on several days of historical data corresponding to multiple user accounts, thereby obtaining the fluctuation range value corresponding to different evaluation indicators for each day. Using the above method, after obtaining the fluctuation range of different evaluation indicators for each of the other test layers besides the target test layer (such as test layer 1), such as test layer 2 and test layer 3, over several historical days, statistical analysis is also performed on the fluctuation range of different evaluation indicators for each of the other test layers over several historical days to obtain the final fluctuation range value corresponding to different evaluation indicators. This final fluctuation range value can then be used as a threshold condition to determine whether the sampling grouping has ended.

[0133] It's important to note that the bootstrap method described above is a uniform sampling method with replacement from a given dataset. This means that whenever a data sample is selected, it is likely to be selected again and added back to the dataset. The commonly used sampling granularity mentioned above refers to tests at the test layer, and is related to a test of the target application at the test layer. For example, for the target application, if the second-class dataset at the test layer is divided into two data groups, the commonly used sampling granularity is based on these data groups. In this case, when calculating the fluctuation range of different evaluation metrics for each day over several historical days for the test layer, it can be obtained by calculating the different evaluation metrics corresponding to at least two data groups corresponding to the test layer over those historical days. The different evaluation metrics corresponding to a data group can be determined by calculating the different evaluation metrics corresponding to each of the second-class datasets included in that data group.

[0134] In another embodiment, the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups can be considered to meet the difference requirement when the weighted sum of the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set sum condition. Specifically, when calculating the weighted sum of the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups, the weights assigned to different indicators can be configured by the client user according to the importance level of different indicators when setting up the test task. The weights between different indicators can be the same or different, and this is not limited here. Furthermore, the above-mentioned sum condition can be determined based on the number of executions corresponding to the current sampling group statistical analysis. For example, if the current analysis is the first sampling group statistical analysis, the sum condition can be a set sum threshold. If the weighted sum of the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups (hereinafter referred to as the first weighted sum for ease of description) is not greater than the sum threshold, the first weighted sum is considered to meet the set sum condition. If the current analysis is not the first sampling group statistical analysis, the sum condition can be the weighted sum of the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups, determined based on the analysis results obtained from the previous sampling group statistical analysis (hereinafter referred to as the second weighted sum for ease of description). In this case, if the first weighted sum is less than the second weighted sum, the first weighted sum is considered to meet the set sum condition.

[0135] In sections 10323 and 10324 above, if the statistical analysis of the current sampling group does not meet the termination condition, the recorded analysis results can be used to participate in the determination of the sum value condition in section 10322 above. If the statistical analysis of the current sampling group meets the termination condition, it indicates that there are minimal differences between the currently determined second number of data groups, which can be ignored. Subsequently, these second number of data groups can be used for different test versions of the target application. Based on the user account feedback data in the data groups corresponding to each test version, the advantages and disadvantages of different test versions can be analyzed and evaluated, thus obtaining more realistic and accurate test results. The second number of data groups determined above is at least two groups, with one group serving as the control group and at least one other group serving as the test group.

[0136] Figure 3 The diagram illustrates the case where test layer 1 is used as the target test layer, and the 1000 second-class datasets contained in the target test layer are divided into two data groups.

[0137] The process of grouping the second type of dataset contained in the target test layer described above can be simply described as follows: Figure 4 The process is as follows: Statistical analysis of historical data for several days corresponding to each user account in 100% of user accounts is performed to determine the indicator data corresponding to the first number of first-class datasets and normalize them; Based on the normalized indicator data corresponding to the first number of first-class datasets, a weighted summation calculation is performed between the indicator data corresponding to the corresponding first-class datasets to obtain the indicator data corresponding to each second-class dataset in the multiple second-class datasets included in the target test layer; Based on the indicator data corresponding to each second-class dataset, a sampling group statistical analysis and evaluation is performed on the multiple second-class datasets included in the target test layer using a method such as MC (i.e., the statistical grouping model mentioned above) to determine whether the sampling group statistical analysis meets the termination condition, and then, if the termination condition is met, at least two data groups are obtained based on the analysis results corresponding to the termination condition.

[0138] As can be seen from the above, the technical solution provided in this embodiment uses a sampling grouping method combining statistical and iterative sampling to group the multiple second-type datasets contained in the target test layer. Although the sampling grouping method provided in this embodiment is slightly inferior to the random sampling in existing solutions in terms of grouping timeliness, it is negligible compared to the time cost of restarting the test when AA-diff occurs. Moreover, the technical solution provided in this embodiment allows for the configuration of weights between indicators for each test, in addition to customizable associated evaluation indicators, to distinguish the coreness of different indicators. When sampling is limited, the weights can also be adjusted to ensure the overall effectiveness of unbiased sampling.

[0139] In addition to using the reinforcement learning-based statistical grouping model described above to group multiple second-class datasets of the target test layer, we can also use the attribute information (including historical behavior data) corresponding to multiple user accounts and a pre-built and trained prediction model based on machine learning algorithms to predict the variance, mean, etc. of multiple grouped samples during the test period, thereby obtaining an optimal grouped sample containing at least two data groups with minimal differences between them. The at least two data groups contained in the optimal grouped sample are then used as data groups. The data groups contained in the grouped sample can be obtained by randomly grouping multiple second-class datasets according to a second quantity.

[0140] That is, in another feasible technical solution, the above-mentioned 1032 "sampling and grouping the plurality of second-type datasets according to the second quantity to obtain the second quantity of data groups" can be specifically implemented by the following steps:

[0141] S21. Determine multiple grouped samples; wherein, the data grouping contained in the grouped samples is obtained by randomly grouping the multiple second-class datasets according to the second quantity;

[0142] S22. When testing the target application using each group of samples, a prediction model is used to predict the second indicator data corresponding to the second number of data groups contained in each group of samples, and the prediction result is obtained.

[0143] S23. Based on the prediction results, a target group sample is determined from the plurality of group samples, and the second number of data groups contained in the target group sample are taken as the second number of data groups obtained.

[0144] In S21 and S22 above, a prediction model can be used to predict the second indicator data corresponding to the second number of data groups contained in each group sample when testing the target application, based on the attribute information of each user account in multiple user accounts; wherein, the prediction model can be, but is not limited to, constructed based on deep learning algorithms, neural network algorithms, etc. Based on this, one possible implementation scheme of step S22 above is:

[0145] S221. Based on the attribute information of the multiple user accounts, perform the prediction using the prediction model to predict the second indicator data corresponding to the second number of data groups contained in each group sample.

[0146] In S23 above, based on the prediction results, a corresponding second indicator data can be selected from multiple grouped samples as the optimal grouped sample, which is then used as the target grouped sample. Thus, the second number of data groups contained in the target grouped sample are the obtained second number of data groups.

[0147] The above-mentioned machine learning method is used to group the multiple second-class datasets contained in the target test. Unlike traditional grouping methods, it predicts the distribution or distribution properties of the differences of the inherent indicators (such as mean, variance, etc.) of each of the at least two data groups contained in each group during the test period (i.e. the future), so as to ensure that the grouping results with good consistency can be obtained.

[0148] In summary, the technical solution provided in this embodiment, after obtaining the test task for the target application, first allocates multiple user accounts to a first number of first-type datasets based on the attribute information of multiple user accounts. During this allocation process, the same user account is within one first-type dataset, and the attribute distribution of user accounts in different first-type datasets is consistent. This effectively ensures the representativeness of all user accounts contained in each first-type dataset to the overall user accounts, helping to reduce system errors. Furthermore, based on the first number of first-type datasets, multiple second-type datasets containing at least two first-type datasets are generated. Then, these second-type datasets are grouped to obtain corresponding data groups. This grouping method reduces the error of random grouping and effectively ensures that different data groups have minimal differences. This allows for more realistic and accurate test results when subsequently testing the version of the target application using these data groups, avoiding redundant testing and improving testing efficiency.

[0149] In summary, the technical solution provided in this application has at least the following beneficial effects:

[0150] 1. Reduce systematic errors: By using cross-stratified sampling to distribute multiple user accounts into a first number of first-class datasets, the consistency of user account performance in each first-class dataset can be guaranteed. This also helps to improve the consistency between the second-class datasets when generating multiple corresponding second-class datasets based on the first number of first-class datasets, thereby reducing systematic errors.

[0151] 2. Reduce random error: By using statistical methods in reinforcement learning (such as the MC method) or machine learning methods to group multiple second-class datasets, the fluctuation range of user accounts between groups can be reduced, thereby reducing the width of the confidence interval, improving the sensitivity of test evaluation, and reducing random error.

[0152] 3. Improved testing efficiency: The two sampling methods in this application (i.e., cross-stratified sampling and group sampling implemented using statistical methods in machine learning or reinforcement learning) are both unbiased sampling. Both unbiased sampling methods can reduce AA-dif, which can effectively reduce the probability of encountering significant differences in indicators during testing, thereby avoiding retesting. At the same time, the unbiased sampling based on the combination of statistics and machine learning iteration, which reduces random error, can directly remove the preAA stage, thereby reducing the testing cycle.

[0153] Based on the above content, a data processing system will be introduced below. Figure 5a The diagram shown is a structural schematic of a data processing system provided in one embodiment of this application. Figure 5a As shown, the data processing system includes:

[0154] Client 100 is used to send test tasks for the target application to the server;

[0155] Server 200 is configured to, upon receiving the test task, allocate multiple user accounts to a first number of first-type datasets based on the attribute information of multiple user accounts; wherein, the same user account is within one first-type dataset, and the attribute distribution of user accounts contained in different first-type datasets is consistent; based on the first number of first-type datasets, generate multiple second-type datasets; each second-type dataset contains at least two first-type datasets; group the multiple second-type datasets to obtain data groups, which are used to test the version under test of the target application.

[0156] In specific implementation, the aforementioned client 100 may, but is not limited to, executing a test task for the target application sent to the service after responding to a user-triggered operation. For example... Figure 5b In the example shown, in the technical solution provided in this embodiment, the client 100 can be, but is not limited to, a desktop computer, tablet computer, smartphone, or smart wearable device (such as a smartwatch or glasses). The server 200 can be, but is not limited to, a single server, a virtual server deployed on a server or server cluster, or a collection of computers based on cloud computing (i.e., the cloud). The cloud consists of a large number of hosts or network servers based on cloud computing, and cloud computing is a type of distributed computing that consists of a super virtual computer composed of a group of loosely coupled computers.

[0157] The communication method between the client 100 and the server 200 can be wireless communication, which can be, but is not limited to, Ethernet, ZigBee, mobile network-based wireless communication, etc.; among which, the mobile network standard can be, but is not limited to, any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, UTMS), 4G (LTE), 4G+ (LTE+), 5G, WiMax, etc.

[0158] It should be noted that any steps in the data processing system provided in this application that are not described in detail can be found in the corresponding content of the above embodiments, and will not be repeated here. Furthermore, the data processing system provided in this application may include other parts or all of the steps in the above embodiments in addition to the steps described above; details can be found in the corresponding content of the above embodiments, and will not be repeated here.

[0159] Furthermore, Figure 6 A schematic diagram of a data processing system according to an embodiment of this application is also shown. This data processing system is described from the perspective of the architecture of a test platform, which can be installed on... Figure 5a or Figure 5b The server 200 is shown. (For example...) Figure 6 As shown, the data processing system includes: an application layer 201 and a test processing layer 202; wherein,

[0160] Application layer 201 is used to provide test tasks of the target application to test processing layer 202;

[0161] The test processing layer 202 is used to, after obtaining the test task from the application layer, allocate multiple user accounts to a first number of first-type datasets based on the attribute information of multiple user accounts; wherein, the same user account is in one first-type dataset, and the attribute distribution of user accounts contained in different first-type datasets is consistent; based on the first number of first-type datasets, a second-type dataset is generated; the second-type dataset contains at least two first-type datasets; the multiple second-type datasets are grouped to obtain data groups; the data groups are used to test the version under test of the target application.

[0162] The primary function of the application layer 201 is to provide an interactive interface, accessible to client users via their clients. This interface allows them to configure test tasks for the target application. In other words, the test tasks provided by the application layer 201 to the test processing layer can be configured by the client user through the interactive interface provided by the application layer 201. In practice, test tasks may include, but are not limited to, basic test information, test version information, relevant metrics, and target audience. For a detailed description of test tasks, please refer to the relevant content above. In addition to providing an interactive interface for client users to configure test tasks, the application layer 201 also provides functions for client users to view test results and other features. For other functions that the application layer 201 can implement, please refer to the existing content.

[0163] The main functions implemented by the test processing layer 202 are test permission management, test management (such as distributing multiple user accounts to a first number of first-type datasets as provided in this embodiment), and retrieval of test results for the target application.

[0164] Furthermore, the data processing system provided in this embodiment may also include:

[0165] Data layer 203 is used to interact with the database for data storage and retrieval;

[0166] The test processing layer 202 is also used to retrieve the plurality of user accounts from the database through the data layer; and is also used to group and store the data in the database through the data layer.

[0167] It should be noted that any steps in the data processing system provided in this embodiment that are not described in detail can be found in the corresponding contents of the above embodiments, and will not be repeated here. Furthermore, the data processing system provided in this embodiment may include other parts or all of the steps in the above embodiments in addition to the steps described above; for details, please refer to the corresponding contents of the above embodiments, and will not be repeated here.

[0168] Figure 7 This diagram illustrates a structural block diagram of a data processing apparatus according to an embodiment of the present application. The data processing apparatus includes: an allocation module 31, an allocation module 32, and a grouping module 33; wherein,

[0169] The allocation module 31 is used to receive a test task for the target application and, based on the attribute information of multiple user accounts, allocate the multiple user accounts to a first number of first-type datasets; wherein, the same user account is in one first-type dataset, and the attribute distribution of user accounts contained in different first-type datasets is consistent;

[0170] The generation module 32 is used to generate multiple second-class datasets based on the first number of first-class datasets; the second-class datasets contain at least two first-class datasets;

[0171] Grouping module 33 is used to group the plurality of second-type datasets to obtain data groups; the data groups are used to test the version of the target application to be tested.

[0172] Furthermore, the grouping module 33 described above, when grouping the plurality of second-type datasets to obtain data groups, is specifically used to: determine a second number of groups; wherein the second number is a positive integer greater than 1; and group the plurality of second-type datasets according to the second number to obtain the second number of data groups.

[0173] Further, the grouping module 33 described above, when used to group the plurality of second-type datasets according to the second quantity to obtain the second quantity of data groups, is specifically used for: determining the first indicator data corresponding to each of the plurality of second-type datasets; based on the first indicator data, using a statistical grouping model to perform sampling group statistical analysis on the plurality of second-type datasets according to the second quantity, obtaining analysis results containing the data groups to which each second-type dataset belongs; when it is determined that the analysis results do not meet the termination condition of the sampling group statistical analysis, recording the analysis results, and returning to execute the step of performing sampling group statistical analysis on the plurality of second-type datasets according to the first indicator data and using a statistical grouping model according to the second quantity; when it is determined that the analysis results meet the termination condition, determining the second quantity of data groups based on the analysis results.

[0174] Furthermore, the above analysis results satisfy the termination condition including the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups, thus satisfying the difference requirement;

[0175] The second indicator data is determined based on the first indicator data corresponding to all second-class datasets included in the data group, and the first indicator data is determined based on the historical data of all user accounts included in the second-class dataset; the requirement to meet the difference includes at least one of the following: the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set threshold condition, and the weighted sum of the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set sum condition.

[0176] Further, the grouping module 33 described above, when used to group the plurality of second-type datasets according to the second quantity to obtain the second quantity of data groups, is specifically used for: determining a plurality of grouped samples; wherein, the data groups contained in the grouped samples are obtained by randomly grouping the plurality of second-type datasets according to the second quantity; when using each grouped sample to test the version to be tested of the target application, using a prediction model to predict the second indicator data corresponding to the second quantity of data groups contained in each grouped sample, and obtaining a prediction result; based on the prediction result, determining a target grouped sample from the plurality of grouped samples, and using the second quantity of data groups contained in the target grouped sample as the obtained second quantity of data groups.

[0177] Furthermore, the aforementioned grouping module 33, when used to predict the second indicator data corresponding to the second number of data groups contained in each group sample by using a prediction model to obtain prediction results when testing the test version of the target application using each group sample, is specifically used to: based on the attribute information of multiple user accounts, execute the prediction of the second indicator data corresponding to the second number of data groups contained in each group sample by using a prediction model to obtain prediction results.

[0178] Further, the aforementioned multiple user accounts include multiple first user accounts; the first user accounts are user accounts whose attribute information meets preset conditions, wherein the attribute information meeting the preset conditions includes at least one of the following: a time attribute value greater than or equal to a preset duration, and an activity attribute value greater than or equal to a preset activity value. Accordingly, the aforementioned allocation module 31, when used to allocate the multiple user accounts to a first number of first-type datasets based on the attribute information of the multiple user accounts, is specifically used to: perform stratification processing on the multiple first user accounts based on the attribute information of the multiple first user accounts to obtain multiple sampling layers; wherein the user attribute information of different first user accounts on the same sampling layer is consistent; determine the sampling ratio corresponding to each sampling layer in the multiple sampling layers; and perform stratified sampling on the first user accounts on the multiple sampling layers according to the sampling ratio, so as to distribute the multiple first user accounts to a first number of first-type datasets.

[0179] Furthermore, the aforementioned plurality of user accounts also includes a second user account, which is a user account whose attribute information does not meet the preset conditions; correspondingly, the allocation module 31, when used to allocate the plurality of user accounts to a first number of first-type datasets according to the attribute information of the plurality of user accounts, specifically further includes: determining the dataset features corresponding to each first-type dataset based on the attribute information of the first user accounts contained in each of the first number of first-type datasets; and selecting, according to the dataset features, a first-type dataset matching the first user account from the first number of first-type datasets as the target first-type dataset allocated to the first user account.

[0180] It should be noted that the data processing device provided in this embodiment can achieve the above-mentioned... Figure 2 The technical solutions described in the data processing method embodiments shown above, and the specific implementation principles of each module or unit can be found in the above description. Figure 2 The relevant content in the data processing method embodiments shown will not be repeated here.

[0181] Figure 8 A schematic diagram of the structure of an electronic device provided according to an embodiment of this application is shown. Figure 8 As shown, the electronic device includes a memory 41 and a processor 42. The memory 41 stores one or more computer instructions; the processor 42, coupled to the memory 41, is used to execute one or more computer instructions (such as computer instructions for implementing data storage logic) to implement the above-mentioned... Figure 2 The steps in the illustrated data processing method embodiment are shown.

[0182] The memory 41 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0183] Furthermore, such as Figure 8 As shown, the electronic device also includes other components such as a communication component 43, a power supply component 44, and an audio component 45. Figure 8 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 8 The components shown.

[0184] Another embodiment of this application provides a computer program product (not shown in the accompanying drawings). This computer program product includes a computer program or instructions that, when executed by a processor, enable the processor to perform the steps described in the data processing method embodiments.

[0185] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, can implement the data processing method steps or functions provided in the above embodiments.

[0186] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing method, characterized in that, include: Upon receiving a test task for the target application, the user accounts are assigned to a first number of first-class datasets based on their attribute information; wherein, the same user account is in one first-class dataset, and the attribute distribution of user accounts in different first-class datasets is consistent. Based on a first number of first-type datasets, multiple second-type datasets are generated; each second-type dataset contains at least two first-type datasets. Determine the second number of groups; where the second number is a positive integer greater than 1; The datasets of the second type are grouped according to the second number to obtain the second number of data groups; the data groups are used to test the version of the target application to be tested.

2. The method according to claim 1, characterized in that, The step of grouping multiple datasets of the second type according to the second quantity to obtain the second quantity of data groups includes: Determine the first indicator data corresponding to each of the multiple datasets of the second type; Based on the first indicator data, a statistical grouping model is used to perform sampling grouping statistical analysis on multiple second-class datasets according to the second quantity, and an analysis result containing the data group to which each second-class dataset belongs is obtained. When it is determined that the analysis results do not meet the termination conditions of the sampling group statistical analysis, the analysis results are recorded, and the process of performing the step of sampling group statistical analysis on multiple second-class datasets according to the second quantity based on the first indicator data and using the statistical grouping model is returned; When it is determined that the analysis results meet the termination condition, the second number of data groups are determined based on the analysis results.

3. The method according to claim 2, characterized in that, The analysis results satisfy the termination conditions, including: The differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meet the difference requirements; The second indicator data is determined based on the first indicator data corresponding to all second-class datasets included in the data group, and the first indicator data is determined based on the historical data of all user accounts included in the second-class dataset. The requirement to meet the difference includes at least one of the following: The difference between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set threshold condition; The weighted sum of the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set sum condition.

4. The method according to claim 1, characterized in that, The step of grouping multiple datasets of the second type according to the second quantity to obtain the second quantity of data groups includes: Multiple grouped samples are determined; wherein the data grouping contained in the grouped samples is obtained by randomly grouping multiple datasets of the second type according to the second quantity; When testing the version of the target application using each group of samples, a prediction model is used to predict the second indicator data corresponding to the second number of data groups contained in each group of samples, and the prediction results are obtained. Based on the prediction results, a target group sample is determined from the plurality of group samples; The second number of data groups contained in the target group sample are taken as the second number of data groups obtained.

5. The method according to claim 4, characterized in that, The method of using a prediction model to predict the second indicator data corresponding to the second number of data groups contained in each group sample, and obtaining the prediction results, includes: Based on the attribute information of multiple user accounts, the prediction model is used to predict the second indicator data corresponding to the second number of data groups contained in each group sample, and the prediction result is obtained.

6. The method according to any one of claims 1 to 5, characterized in that, The plurality of user accounts includes a plurality of first user accounts, wherein the first user accounts are user accounts whose attribute information meets preset conditions; as well as The step of assigning multiple user accounts to a first number of first-class datasets based on the attribute information of multiple user accounts includes: Based on the attribute information of each first user account, the multiple first user accounts are stratified to obtain multiple sampling layers; wherein, the attribute distribution of different first user accounts in the same sampling layer is consistent. Determine the sampling ratio corresponding to each of the multiple sampling layers; According to the sampling ratio, the first user accounts on multiple sampling layers are stratified and sampled in order to distribute the multiple first user accounts into a first number of first type datasets.

7. The method according to claim 6, characterized in that, The plurality of user accounts also includes a second user account, which is a user account whose attribute information does not meet the preset conditions; as well as The step of assigning multiple user accounts to a first number of first-class datasets based on the attribute information of multiple user accounts further includes: Based on the attribute information of the first user accounts contained in each of the first number of first type datasets, determine the dataset features corresponding to each first type dataset; Based on the characteristics of the dataset, the first type of dataset that matches the second user account is selected from a first number of first type datasets, and is used as the target first type dataset for the second user account.

8. A data processing apparatus, characterized in that, include: The allocation module is used to receive a test task for a target application and, based on the attribute information of multiple user accounts, allocate the multiple user accounts to a first number of first-type datasets; wherein, the same user account is in one first-type dataset, and the attribute distribution of user accounts contained in different first-type datasets is consistent; The generation module is used to generate multiple second-class datasets based on the first number of first-class datasets; the second-class datasets contain at least two first-class datasets; A grouping module is used to determine a second number of groups; wherein the second number is a positive integer greater than 1; multiple datasets of the second type are grouped according to the second number to obtain the second number of data groups; the data groups are used to test the version to be tested of the target application.

9. The apparatus according to claim 8, characterized in that, The grouping module, when used to group multiple datasets of the second type according to the second quantity to obtain the second quantity of data groups, is specifically used for: Determine the first indicator data corresponding to each of the multiple datasets of the second type; Based on the first indicator data, the statistical grouping model is used to perform sampling grouping statistical analysis on multiple second-class datasets according to the second quantity, and the analysis results containing the data groups to which each second-class dataset belongs are obtained. When it is determined that the analysis results do not meet the termination conditions of the sampling group statistical analysis, the analysis results are recorded, and the process of performing the step of sampling group statistical analysis on multiple second-class datasets according to the second quantity based on the first indicator data and using the statistical grouping model is returned; When it is determined that the analysis results meet the termination condition, the second number of data groups are determined based on the analysis results.

10. The apparatus according to claim 9, characterized in that, The analysis results satisfy the termination conditions, including: The differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meet the difference requirements; The second indicator data is determined based on the first indicator data corresponding to all second-class datasets included in the data group; the first indicator data is determined based on the historical data of all user accounts included in the second-class dataset. The requirement to meet the difference includes at least one of the following: the difference between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set threshold condition; the weighted sum of the differences between the second indicator data corresponding to each pair of data groups in the second number of data groups meets the set sum condition.

11. The apparatus according to claim 8, characterized in that, The grouping module, when used to group multiple datasets of the second type according to the second quantity to obtain the second quantity of data groups, is specifically used for: Multiple grouped samples are determined; wherein the data grouping contained in the grouped samples is obtained by randomly grouping multiple datasets of the second type according to the second quantity; When testing the version of the target application using each group of samples, a prediction model is used to predict the second indicator data corresponding to the second number of data groups contained in each group of samples, and the prediction results are obtained. Based on the prediction results, a target group sample is determined from the plurality of group samples; The second number of data groups contained in the target group sample are taken as the second number of data groups obtained.

12. The apparatus according to claim 11, characterized in that, The grouping module, when used to predict the second indicator data corresponding to the second number of data groups contained in each group sample using a prediction model to obtain the prediction result, is specifically used for: Based on the attribute information of multiple user accounts, the prediction model is executed to predict the second indicator data corresponding to the second number of data groups contained in each group sample, and the prediction result is obtained.

13. The apparatus according to any one of claims 8 to 12, characterized in that, The plurality of user accounts includes a plurality of first user accounts, wherein the first user accounts are user accounts whose attribute information meets preset conditions; the allocation module, when used to allocate the plurality of user accounts to a first number of first-type datasets according to the attribute information of the plurality of user accounts, is specifically used for: Based on the attribute information of each first user account, the multiple first user accounts are stratified to obtain multiple sampling layers; wherein, the attribute distribution of different first user accounts in the same sampling layer is consistent. Determine the sampling ratio corresponding to each of the plurality of sampling layers; According to the sampling ratio, the first user accounts on the multiple sampling layers are stratified and sampled in order to distribute the multiple first user accounts into a first number of first type datasets.

14. The apparatus according to claim 13, characterized in that, The plurality of user accounts also includes a second user account; the second user account is a user account whose attribute information does not meet the preset conditions. The allocation module, when used to allocate multiple user accounts to a first number of first-type datasets based on the attribute information of multiple user accounts, specifically further includes: Based on the attribute information of the first user accounts contained in each of the first number of first type datasets, determine the dataset features corresponding to each first type dataset; Based on the characteristics of the dataset, the first type of dataset that matches the second user account is selected from a first number of first type datasets, and is used as the target first type dataset for the second user account.

15. A data processing system, characterized in that, include: The client is used to send test tasks for the target application to the server. The server, upon receiving the test task, assigns multiple user accounts to a first number of first-type datasets based on their attribute information; wherein the same user account is within one first-type dataset, and the attribute distribution of user accounts in different first-type datasets is consistent; generates multiple second-type datasets based on the first number of first-type datasets; each second-type dataset contains at least two first-type datasets; determines a second number of groups; wherein the second number is a positive integer greater than 1; groups the multiple second-type datasets according to the second number to obtain the second number of data groups; the data groups are used to test the version under test of the target application.

16. A data processing system, characterized in that, include: The application layer is used to provide test tasks for the target application to the test processing layer; A test processing layer is used to, after obtaining the test task from the application layer, allocate multiple user accounts to a first number of first-type datasets based on the attribute information of multiple user accounts; wherein, the same user account is in one first-type dataset, and the attribute distribution of user accounts contained in different first-type datasets is consistent; based on the first number of first-type datasets, generate multiple second-type datasets; the second-type datasets contain at least two first-type datasets; determine a second number of groups; wherein, the second number is a positive integer greater than 1; group the multiple second-type datasets according to the second number to obtain the second number of data groups; the data groups are used to test the version under test of the target application.

17. The system according to claim 16, characterized in that, Also includes: The data layer is used to interact with the database for data storage and retrieval; The test processing layer is further configured to obtain attribute information of the multiple user accounts from the database through the data layer; and to group and store the data in the database through the data layer.

18. An electronic device, characterized in that, include: Memory and processor, among which, The memory is used to store one or more computer programs; The processor, coupled to the memory, is configured to execute the one or more computer programs stored in the memory to implement the steps of the method according to any one of claims 1 to 7.

19. A computer-readable medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 7.

20. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 7.

Images