Data processing system and method, data testing system, data system

Abstract

The embodiment of the present application provides a data processing system and method, a data testing system, and a data system. The data processing system includes: a data loading layer, which is used to determine a target test type from at least one candidate test type; load the test data to be tested provided by the test user according to the target test type; load the configuration information corresponding to the test system; a data processing layer, which is used to parse the configuration information to obtain a parsing result; based on the parsing result, select a target test algorithm from at least one test algorithm; a data analysis layer, which is used to call the target test algorithm, perform data analysis on the data to be tested, and obtain a data analysis result. The embodiment of the present application improves the utilization efficiency of the test system.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing system and method, a data testing system, and a data system. Background Art

[0002] The testing system can randomly test multiple products with similar attributes in the same time dimension to make decisions on multiple products. For example, the more common A / B test is to let multiple target users with similar attributes randomly access more than two versions of web pages or applications, collect user experience data or business data, obtain test results through evaluation and analysis of test data such as user experience data and / or business data, and use the test results to make version decisions.

[0003] Existing testing systems, such as A / B testing systems, are mostly web pages developed by developers based on testing engines and provide testing services to testing users. The testing system can obtain multiple versions of interface screenshots, URL links or applications specified by the platform uploaded by testing users, and then the testing system completes the test based on the testing algorithm configured in its testing engine.

[0004] From the above description, it can be seen that the existing test system can only complete the test according to the test service provided by the test engine, resulting in low utilization of the test system. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a data processing system and method, a data testing system, and a data system to solve the technical problem in the prior art that the test system can only complete the test according to the test service provided by the test engine, resulting in low utilization of the test system.

[0006] In a first aspect, an embodiment of the present application provides a data processing system, including:

[0007] The data loading layer is used to determine the target test type from at least one candidate test type; load the test data provided by the test user according to the target test type; and load the configuration information corresponding to the test system;

[0008] The data processing layer is used to parse the configuration information to obtain a parsing result; based on the parsing result, select a target test algorithm from at least one test algorithm;

[0009] The data analysis layer is used to call the target test algorithm, perform data analysis on the data to be tested, and obtain data analysis results.

[0010] In a second aspect, an embodiment of the present application provides a data testing system, including: a testing subsystem and a data processing subsystem;

[0011] The test subsystem is used to: display the target test type to the test user; obtain the test data provided by the test user according to the target test type; and provide the test data to the data processing system;

[0012] The data processing subsystem includes: a data loading layer, which is used to determine a target test type from at least one candidate test type; load the test data to be tested provided by the test user according to the target test type; load the configuration information corresponding to the test subsystem; a data processing layer, which is used to parse the configuration information to obtain a parsing result; based on the parsing result, select a target test algorithm from at least one test algorithm; a data analysis layer, which is used to call the target test algorithm, perform data analysis on the data to be tested, and obtain a data analysis result.

[0013] In a third aspect, a data system is provided, including: a development subsystem, a test subsystem, and a data processing subsystem;

[0014] The development subsystem is used to: detect a target test type selected by a development user from at least one candidate test type; provide the target test type to the data processing system; generate the test subsystem and a data test system corresponding to the data processing subsystem;

[0015] The test subsystem is used to: display the target test type to the test user; obtain the test data provided by the test user according to the target test type; and provide the test data to the data processing system;

[0016] The data processing subsystem includes: a data loading layer, which is used to determine a target test type from at least one candidate test type; load the test data to be tested provided by the test user according to the target test type; load the configuration information corresponding to the test subsystem; a data processing layer, which is used to parse the configuration information to obtain a parsing result; based on the parsing result, select a target test algorithm from at least one test algorithm; a data analysis layer, which is used to call the target test algorithm, perform data analysis on the data to be tested, and obtain a data analysis result.

[0017] A fourth aspect provides a data processing method, comprising:

[0018] determining a target test type from at least one candidate test type;

[0019] Load the test data provided by the test user according to the target test type;

[0020] Load the configuration information corresponding to the test system;

[0021] Parsing the configuration information to obtain a parsing result;

[0022] Based on the analysis result, selecting a target test algorithm from the at least one test algorithm;

[0023] The target test algorithm is called to perform data analysis on the data to be tested to obtain data analysis results.

[0024] In the embodiment of the present application, the data processing system can actually be configured as a test engine, and can include: a data loading layer, which can select a target test type from at least one candidate test type. By providing at least one candidate test type, the test types that the test system can support can be increased, and the actual testable range can be expanded. The test user loads the test data provided by the target test type and the configuration information corresponding to the load test system. The data loading layer independently completes the loading of the test data and the configuration information, realizes modular data processing, and improves data processing efficiency. The data processing layer can parse the configuration information to obtain the parsing result, and then selects the target test algorithm from at least one test algorithm based on the parsing result. By supporting the selection of at least one test algorithm, the test system can support more test methods and expand the application scope of the test system. The target test algorithm is called in the data analysis layer to perform data analysis on the test data, obtain the data analysis result, and complete the test of the test system. By increasing the supported test types and expanding the supported test methods, the use function of the test engine is expanded, thereby improving the use scope of the test system, so that the test system supports multiple test types and multiple test methods, and improves the utilization efficiency of the test system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A schematic diagram of the structure of an embodiment of a data processing system provided in an embodiment of the present application;

[0027] Figure 2 A structural diagram of another embodiment of a data processing system provided in an embodiment of the present application;

[0028] Figure 3A schematic diagram of the structure of an embodiment of a data system provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of the structure of an embodiment of a data testing system provided in an embodiment of the present application;

[0030] Figure 5a An example diagram of a data testing system provided in an embodiment of the present application;

[0031] Figure 5b An example diagram of a data system provided in an embodiment of the present application;

[0032] Figure 6 A flowchart of an embodiment of a data processing method provided in an embodiment of the present application;

[0033] Figure 7 A flowchart of another embodiment of a data processing method provided in an embodiment of the present application;

[0034] Figure 8 A flowchart of another embodiment of a data processing method provided in an embodiment of the present application;

[0035] Fig. 9 A schematic diagram of the structure of an embodiment of a computing device provided in an embodiment of the present application;

[0036] Fig.10 A schematic diagram of the structure of an embodiment of an electronic device provided in an embodiment of the present application;

[0037] Fig.11 A schematic structural diagram of another embodiment of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0040] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0041] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to identifying", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is identified" may be interpreted as "when determining" or "in response to determining" or "when identifying (stated condition or event)" or "in response to identifying (stated condition or event)", depending on the context.

[0042] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0043] The technical solution of the embodiment of the present application can be applied in testing scenarios. By providing a test engine that supports multiple test types and multiple test algorithms, developers can develop or use the test system in an extensible manner, thereby expanding the functionality of the test engine and improving the utilization rate of the test system.

[0044] At present, the test system can be a test website developed by the developer based on the test engine. The website page of the test system can be provided to the test user, and the test user can upload the web page or application to be tested on the web page, as well as the test data generated when the target user visits the web page or application. The test system performs data analysis on the test data based on the test algorithm in its own test engine to obtain the test results. The test engine can only complete the test service based on the test algorithm it provides, which results in a relatively limited application scope of the test system and a low utilization rate of the test system.

[0045] In the embodiment of the present application, the data processing system can actually be configured as a test engine, and can include: a data loading layer, which can determine the target test type from at least one candidate test type. By providing at least one candidate test type, the test types that the test system can support can be increased, and the actual test scope can be expanded. The test user loads the test data provided by the target test type and the configuration information corresponding to the load test system. The data loading layer independently completes the loading of the test data and the configuration information, realizes modular data processing, and improves data processing efficiency. The data processing layer can parse the configuration information to obtain the parsing result, and then select the target test algorithm from at least one test algorithm based on the parsing result. By supporting the selection of at least one test algorithm, the test system can support more test methods and expand the application scope of the test system. The target test algorithm is called in the data analysis layer to perform data analysis on the test data, obtain the data analysis result, and complete the test of the test system. By increasing the supported test types and expanding the supported test methods, the use function of the test engine is expanded, thereby improving the use scope of the test system, so that the test system supports multiple test types and multiple test methods, and improves the utilization efficiency of the test system.

[0046] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0047] like Figure 1 As shown, it is a structural diagram of a data processing system provided in an embodiment of the present application, and the data processing may include: a data loading layer 10, a data processing layer 20, and a data analysis layer 30.

[0048] The data processing system provided in the embodiment of the present application can be applied to various testing scenarios, such as medical testing scenarios, Internet testing scenarios, industrial testing scenarios, or other scenarios that require multi-version testing, etc. This embodiment does not impose too many limitations on the application scenarios.

[0049] In this embodiment, the data loading layer 10 can be used to: determine a target test type from at least one candidate test type; load the test data provided by the test user according to the target test type; and load the configuration information corresponding to the test system.

[0050] The data processing layer 20 can be used to: parse the configuration information to obtain a parsing result; and select a target test algorithm from at least one test algorithm based on the parsing result.

[0051] The data analysis layer 30 can be used to call a target test algorithm, perform data analysis on the test data, and obtain data analysis results.

[0052] Among them, at least one candidate test type may include: a flow test type and a batch test type; at least one test algorithm may include: a hypothesis testing algorithm, a sequential detection algorithm, a controlled variance test algorithm and / or a non-parametric test algorithm. The target test type may be a flow test type or a batch test type.

[0053] In the embodiment of the present application, the data processing system can actually be configured as a test engine, and can include: a data loading layer, which can determine the target test type from at least one candidate test type. By providing at least one candidate test type, the test types that the test system can support can be increased, and the actual test scope can be expanded. The test user loads the test data provided by the target test type and the configuration information corresponding to the load test system. The data loading layer independently completes the loading of the test data and the configuration information, realizes modular data processing, and improves data processing efficiency. The data processing layer can parse the configuration information to obtain the parsing result, and then select the target test algorithm from at least one test algorithm based on the parsing result. By supporting the selection of at least one test algorithm, the test system can support more test methods and expand the application scope of the test system. The target test algorithm is called in the data analysis layer to perform data analysis on the test data, obtain the data analysis result, and complete the test of the test system. By increasing the supported test types and expanding the supported test methods, the use function of the test engine is expanded, thereby improving the use scope of the test system, so that the test system supports multiple test types and multiple test methods, and improves the utilization efficiency of the test system.

[0054] In a possible design, the data loading layer may determine the target test type from at least one candidate test type and specifically include:

[0055] Detect the test environment of the test system.

[0056] A target test type matching the test environment is determined from at least one candidate test type.

[0057] The data loading layer loading test user provides the test data according to the target test type, including:

[0058] Load test data provided by the user in the test system according to the target test type.

[0059] The configuration information corresponding to the data loading layer loading test system includes:

[0060] Load test users based on configuration information provided by the test system.

[0061] The test system is the current test system that initiates the test. The test environment is associated with the test type.

[0062] The test environment can refer to the description of various software and hardware environments used to complete the test work. In the Internet testing scenario, the test environment can be the software architecture of the overall structure and components of the software. For example, it can include: blink, Flink, ODPS, etc. The data processing system can support multiple test environments. The test environment can be specified by the developer when developing the test system in the development system. Once the test environment is specified, the test environment of the test system is determined, and the data processing system can run in the test environment specified by the developer.

[0063] In one possible design, the test system may store the association relationship between the test environment and the target test type, and the data loading layer in the data processing system may read the association relationship between the test environment and the target test type from the test system. In another possible design, the data loading layer of the data processing system may store the association relationship between the test environment and the target test type, so as to directly read the association relationship between the test environment and the target test type.

[0064] Optionally, the test user may be a user who uses the data processing system in the test system to test the object. For example, when the test system is a blink architecture platform, the test user is a user of the blink platform. The test user may specifically operate the test system to complete the test of the tested object, thereby judging the availability of the test object through the obtained data analysis results and decision analysis results, or judging whether there are significant differences among the tested objects through the decision analysis results when there are multiple test objects, thereby obtaining a test object with better use effect.

[0065] In a possible design, the configuration information may specifically be file information formed by the processing method selected by the test user during the process of processing the test data by the test system. For example, the data processing system may provide multiple test algorithms, and the test user may select a target test algorithm from the multiple test algorithms. The test user's selection information of the target test algorithm is a type of configuration information. In addition, the configuration information may also include: usage information of the test algorithm in the data processing system, decision analysis information, etc.

[0066] In yet another possible design, the configuration information may be file information pre-stored in the test system and formed by the test system's processing method for the data to be tested.

[0067] In order to make the structure of the configuration information clear and easy to use, the configuration information can be defined using a standardized configuration structure or class. The configuration information may include multiple configuration parameters, and different configuration contents are defined by using different configuration parameters. For example, the configuration structure or configuration class may include configuration parameters such as algorithm name, significance level, test type, and sample type to be tested. Specifically, the configuration information can be obtained by setting the configuration parameters as an object. Any configuration parameter may correspond to a corresponding parameter type, and the parameter type may be specifically determined based on the actual test type of the configuration parameter. For example, the algorithm name is mostly identified by a string, and the configuration parameter corresponding to the algorithm name in the configuration parameter may be a string type, and the algorithm name in the configuration parameter may represent the algorithm name of the data processing algorithm specified by the test user. This embodiment does not make too many restrictions on the specific parameter types and meanings of the configuration parameters.

[0068] In order to enrich the data content of the data to be tested, the data to be tested can exist in the form of a data structure object or a data class object. The data structure or data class may include multiple data parameters, such as: user test hypothesis value, observed sample size, sum of observed sample values, sum of squares of observed samples, etc. The data to be tested can be a structure object or class object to be processed. By assigning values ​​to the data parameters in the structure, the corresponding data to be tested can be obtained.

[0069] For ease of understanding, in an actual configuration information example, the configuration information can be defined by the following structure: rawConfig{

[0070] String type algo; / / algorithm name

[0071] Real-valued alpha; / / significance level

[0072] Real value beta; / / Type II error threshold

[0073] String type testingType; / / Test type

[0074] String type testingSample; / / Type of sample population to be tested}.

[0075] For ease of understanding, in an actual example of test data, the test data can be defined by the following structure: rawData{

[0076] String type expectedValue; / / The hypothesis value tested by the user, including single value, multiple value, interval form, as well as the original hypothesis and alternative hypothesis

[0077] Real value sampleSize; / / Observed sample size

[0078] Real value SumX; / / Sum of observed sample values

[0079] Real value SumXSquare; / / Sum of squares of observed values}.

[0080] Of course, the above structures of the configuration information and the data to be tested are merely illustrative and should not constitute a limitation on the specific parameter setting method of the configuration information and the data to be tested in the embodiments of the present application. It should be understood that the configuration parameters of the configuration information and the data parameters of the data to be tested can be increased or decreased, and can be set specifically according to actual usage needs.

[0081] As described above, the configuration information may include an algorithm name parameter, and the parameter data of the parameter may be the algorithm name of the test algorithm specified by the test user. After the configuration information is parsed and the parsing result is obtained, the parsing result may include the target test algorithm specified by the test user. That is, the data processing layer selects the target test algorithm from at least one test algorithm based on the parsing result, which may specifically include: based on the target algorithm name included in the parsing result, determining the target test algorithm corresponding to the target algorithm name from at least one test algorithm.

[0082] The test system can provide the test data provided by the test user according to the target test type and the configuration information input through the test system to the data processing system, so that the data processing system can process the test data and perform configuration analysis on the configuration information.

[0083] The data testing system may include, in addition to the data processing system formed by the data processing system, a testing system, which may detect the data to be tested and the configuration information input by the test user.

[0084] In this embodiment, the data processing system may actually be a test engine. The core of the data testing system composed of the data processing system and the test system is the data processing system, and the operation of the test system depends on the data processing system.

[0085] The data loading layer, data processing layer and data analysis layer in the data processing system are all independent and universal. Each layer can be called or developed independently, data can be transmitted between different layers, and there can be no overlap in processing content between any two layers.

[0086] In this embodiment, the data loading layer may support the loading of at least one candidate test type. At least one candidate test type may be a test type that simulates the use of the test object by the user and generates usage result data. At least one candidate test type may include: a streaming test type and a batch test type. The data of the streaming test type may be streaming data, and the data of the batch test type may be batch data. The target test type may be selected from the streaming test type and the batch test type. The data to be tested corresponding to the target test type may be batch data or streaming data.

[0087] In a possible design, at least one data loading program corresponding to each candidate test type may be configured in the data loading layer. When any candidate test type is determined as a target test type, the data to be tested provided by the test user according to the target test type may be loaded through the loading program of the target test type.

[0088] In practical applications, the data processing system may provide at least one candidate test type to a development user. For example, the data processing system may provide the data processing system to the development user in the form of a development document. The development user may learn at least one candidate test type through the development document and select a target test type from the candidate test type. The development user may select the target test type and at least one test algorithm through the development system. The development system may complete the use configuration of the data processing system based on the development instructions of the development user to obtain a final test system.

[0089] At least one test algorithm can be configured in the data analysis layer, and the at least one test algorithm can be called at any time to perform data analysis processing. At least one test algorithm can be provided by the data processing system or defined in a user-defined manner, so that the data processing system supports multiple test algorithms, improves the scalability of the data processing system, and makes the algorithm selectivity of the test system configured with the data processing system higher, thereby improving the utilization rate of the test system.

[0090] In practical applications, at least one test algorithm may include: one or more of a basic hypothesis testing algorithm, a sequence testing algorithm, a controlled variance test, a non-parametric test, and a personalized testing algorithm.

[0091] Basic hypothesis testing algorithms, for example, may be Z Test algorithms and / or T Test algorithms. Sequential testing algorithms, for example, may be Wald's SPRT (Wald's Sequential probability ratio test), Mixture SPRT (Mixture Sequential probability ratio test) and / or non-parametric sequential testing. Personalized algorithms, for example, may be Control Variate algorithms. Non-parametric tests may include, for example, NHST (Null hypothesis significance testing).

[0092] As an embodiment, the data analysis layer may include an algorithm calling interface, and the algorithm is called by calling the interface. The data analysis layer calls the target test algorithm, performs data analysis on the test data, and obtains the data analysis results, which may include:

[0093] The target test algorithm is called through the algorithm calling interface, and data analysis and processing are performed on the test data to obtain data analysis results.

[0094] The data analysis layer implements the call of the test algorithm through the algorithm call interface, which can realize the interface call of multiple modular test algorithms. Even if the test algorithm changes, it does not affect the algorithm call, thereby improving the portability of the test algorithm in the data processing system.

[0095] like Figure 2 As shown, it is a structural diagram of a data processing system provided in an embodiment of the present application. Data processing may include: a data loading layer 21, a data processing layer 22, a data analysis layer 23 and a decision analysis layer 24.

[0096] In this embodiment, the data loading layer 21 can be used to: determine the target test type from at least one candidate test type based on the test environment of the current test system; load the test data provided by the test user according to the target test type; load the configuration information provided by the test user.

[0097] The data processing layer 22 can be used to: parse the configuration information to obtain a parsing result; and select a target test algorithm from at least one test algorithm based on the parsing result.

[0098] The data analysis layer 23 can be used to call the target test algorithm, perform data analysis on the test data, and obtain data analysis results.

[0099] The decision analysis layer 24 can be used to perform decision analysis on the data analysis results to obtain decision analysis results.

[0100] In practical applications, the decision analysis results obtained by the decision analysis layer can be output to the test user for viewing. In a possible design, the decision analysis layer 24 can include: a decision analysis interface.

[0101] The decision analysis interface can be used to: output decision analysis results to test users.

[0102] The decision analysis layer can perform decision analysis on the data analysis results to obtain decision analysis results. The decision analysis layer can perform decision analysis on the data analysis results through a decision analysis algorithm to obtain a decision analysis result. The decision analysis algorithm can be determined based on the parsing result of the configuration information, that is, the parsing result can include the decision analysis algorithm when the decision analysis layer performs decision analysis on the data analysis results. The decision analysis algorithm can include multiple decision sub-algorithms, so that multiple decision analyses can be performed on the data analysis results to obtain multiple decision result data.

[0103] In a possible design, the decision analysis result may include multiple decision result data, such as: a difference result identifier, a confidence interval, including an upper confidence limit (confidence Upper bound) and a lower confidence limit (confidence Lower bound), a P value (P-value, a parameter used to determine the result of a hypothesis test), Z statistics, and a test result, etc. Among them, the difference result identifier is used to identify whether there is a difference between two or more test data.

[0104] Optionally, at least one of the multiple decision result data in the decision analysis result can be used to comprehensively confirm whether there is a significant difference between the test data corresponding to each of the tested objects, that is, to confirm whether there is a significant difference in the use results between the two or more tested objects. For example, taking the P value as an example, if the P value in the obtained decision analysis result is compared with the predetermined P value, if the P value is greater than the predetermined P value, it can be considered that there is no significant difference between the tested objects.

[0105] The decision layer can output the decision analysis results through the decision analysis interface. The decision analysis results can be notified to the test user in a timely manner. The addition of the decision layer can expand the processing function of the data processing system, making the processing range of the data processing system wider, thereby increasing the system performance of the test system including the data processing system, further expanding the functions of the test system, and improving the utilization efficiency of the test system.

[0106] In order to achieve the acquisition of the data to be tested, modular processing is performed. The actual external interface of the data loading layer can be a data loading interface. When the data loading interface loads the data to be tested, the loading program of the target test type can be called through the data loading interface to load the data to be tested. The data loading layer may include: a data loading interface.

[0107] The data loading layer loads the test data provided by the test user according to the target test type; the configuration information corresponding to the loading test system may include:

[0108] Load the test data provided by the test user according to the target test type through the data loading interface;

[0109] Load the configuration information corresponding to the test system through the data loading interface.

[0110] In this embodiment, different types of data can use their own sub-interfaces to acquire data. The data loading interface may include at least one data loading sub-interface and a configuration loading sub-interface. For example, when at least one candidate test type may include a streaming test type and a batch test type, the streaming data of the streaming test type may be loaded using the streaming data loading sub-interface, and the data of the batch test type may be loaded using the batch data loading sub-interface.

[0111] When the data loading interface loads the test data provided by the test user according to the target test type, it can first determine the target sub-interface corresponding to the target test type from at least one data loading sub-interface, and then load the test data provided by the test user according to the target test type through the target sub-interface.

[0112] When the configuration information provided by the test user is loaded through the data loading interface, the configuration information provided by the test user can be loaded through the configuration loading sub-interface.

[0113] The test environment can be the working environment of the data processing system. After selecting the test environment, the development user can determine the target test type associated with the test environment. By establishing an association relationship between the test environment and the target test type, the test environment and the target test type can be connected, making the determination of the test environment and the target test type more closely associated, thereby quickly determining the target test type and completing the loading of the data to be tested.

[0114] Therefore, as an embodiment, the data loading layer determining the target test type from at least one test type may specifically include: determining the target test type from at least one candidate test type based on the test environment of the current test system.

[0115] The data loading layer determines the target test type from at least one candidate test type based on the test environment of the current test system, and specifically may include:

[0116] Based on the test environment of the current test system, a target test type associated with the test environment is determined from at least one candidate test type.

[0117] In this embodiment, the association between the test environment and the target test type can be established by the development user during the development process. As a possible implementation method, the data loading layer can also be used for:

[0118] Based on the test environment selected by the development user in the development system and the target test type selected from at least one candidate test type, the test environment and the target test type of the test system are determined.

[0119] In practical applications, the data loading layer can also be used for:

[0120] Establish an association between the test environment and the target test type for the test system.

[0121] In this embodiment, the data loading layer can directly establish and store the association between the test environment of the test system and the target test type. In some other embodiments, the association between the test environment of the test system and the target test type can also be established and stored by the test system.

[0122] By establishing an association between the test environment of a test system and its corresponding target test type, it is possible to quickly determine the target test type corresponding to any test system, thereby improving test efficiency.

[0123] The data processing system obtains the test environment and target test type specified by the development user and establishes an association between the two. When the test environment of the test system is detected, the data processing system can quickly determine the target test type, thereby improving the loading efficiency of the data to be tested.

[0124] At least one test algorithm can actually be provided by a development user to the data processing system. As an embodiment, the data analysis layer is also used to: obtain at least one test algorithm determined by the development user in the development system. The development user determines at least one test algorithm in the development system, and the development system can provide the at least one test algorithm to the data processing system.

[0125] Therefore, in some embodiments, the data analysis layer may also be used to:

[0126] Gets the first processing algorithm for the custom input by the development user in the development system.

[0127] Acquire a second processing algorithm selected by a development user from at least one candidate processing algorithm provided by the data processing system;

[0128] At least one test algorithm consisting of the first processing algorithm and the second processing algorithm is determined.

[0129] In practical applications, the second processing algorithm may include one or more. By providing a selection mechanism for at least one candidate processing algorithm and a customization mechanism for a user-defined algorithm, the selectivity and extensibility of at least one test algorithm are enhanced, providing an expansion basis for the data processing system, thereby promoting the improvement of the utilization scope and utilization efficiency of the data processing system.

[0130] In order to achieve unified data processing, in some embodiments, the data processing layer can also be used to:

[0131] Performing data aggregation processing on the data to be tested to obtain first data;

[0132] converting the first data into second data corresponding to a standard data format predefined by the data processing system;

[0133] The data analysis layer calls the target test algorithm to perform data analysis on the test data, and obtains the following data analysis results:

[0134] The target test algorithm is called to perform data analysis on the second data to obtain a data analysis result.

[0135] Performing data aggregation processing on the data to be tested to obtain the first data may specifically include: performing addition calculation, square sum calculation and / or sample quantity calculation processing on the data to be tested to obtain the first data.

[0136] The first data after the aggregation processing of the data to be tested may be of multiple data types. In order to uniformly process the first data, the first data can be converted into second data corresponding to a standard data format predefined by the data processing system. The second data can be uniformly processed by the data processing system, thereby realizing data analysis processing on the data and obtaining data analysis results. Through the uniform processing of the data format, the data processing system can perform the same data analysis processing on different types of first data, thereby realizing uniform analysis.

[0137] The standard data format predefined by the data processing system may be a data type that can be uniformly processed by the data processing system. The standard data format may be determined according to actual usage requirements, for example, a data format defined in the form of a data structure.

[0138] As an embodiment, the data processing system may further include:

[0139] The data output layer is used to output the data analysis results to the test system, so that the test system can output the data analysis results to the test user, so that the test user can confirm the data analysis results.

[0140] The data output layer may be located after the data analysis layer, and may output the data analysis results obtained by the data analysis layer from analyzing the test data, so as to facilitate the test user to confirm the data analysis results.

[0141] Optionally, the test system outputting the data analysis results to the test user may specifically include: outputting the data analysis results through a data analysis interface of the test system so that the test user can view the data analysis results. The data analysis results can be used by the test user to confirm the test analysis results to determine whether to perform subsequent decision analysis.

[0142] At this time, the decision analysis layer performs decision analysis on the data analysis results, and obtaining the decision analysis results may specifically include: detecting the decision confirmation operation performed by the test user on the data analysis results, performing decision analysis on the data analysis results, and obtaining the decision analysis results.

[0143] In actual applications, in addition to outputting the data analysis results of the data analysis layer, the data output layer can also output the decision analysis results of the decision analysis layer. That is, the data output layer is also used to: output the decision analysis results of the decision analysis layer to the test system, so that the test system can output the decision analysis results to the test user, so that the test user can view the decision results.

[0144] In some embodiments, the data loading layer loads the test data provided by the test user according to the target test type and specifically includes:

[0145] determining at least one test object corresponding to a target test type;

[0146] The test data to be tested provided by the test user for at least one test object is loaded.

[0147] The data processing system may support multiple candidate test types, for example, multiple candidate test types may include page type, medical data type, algorithm type, operation plan test, product plan test, etc. The target test type is selected from at least one candidate test type.

[0148] In some embodiments, the selection of the target test type may be determined by the test user, and the development user may develop a test system that provides the multiple candidate test types, so that the test user can select the target test type from at least one candidate test type provided by the test system.

[0149] In some other embodiments, the selection of the target test type may be determined by a development user. The development user may determine the target data type based on at least one candidate data type supported by the data processing system. The development system may send the target data type and the test environment selected by the development user to the data processing system, and the data processing system may associate and save the test environment and the target data type, so as to determine the target data type corresponding to the test environment of the test system when it is detected that the test system initiates a test request through the data processing system.

[0150] A test user can provide test data to at least one test object in the test system.

[0151] At least one test object is an object to be tested, and the test type of the test object is a target test type. The test system can test at least one test object of the same test type, that is, N test objects can be tested simultaneously, where N is a positive integer greater than or equal to 1.

[0152] For example, when N is equal to 1, it is an independent test of a single object, and when N is equal to 2, it is a test of two test objects, which is a more common A / B test. The number of test objects can be determined according to actual test requirements, and the embodiment of the present application does not impose too many restrictions on the number of objects of at least one test object.

[0153] For ease of understanding, the technical solution of the embodiment of the present application is described in detail by taking the number of objects of at least one test object as two as an example. In one embodiment, the data loading layer determines that at least one test object corresponding to the target test type may specifically include:

[0154] Determining a first test object and a second test object corresponding to a target test type;

[0155] The data loading layer loads the test data provided by the user for at least one test object, specifically including:

[0156] Loading first test data provided by a test user for a first test object;

[0157] Loading second test data provided by the test user for the second test object;

[0158] The data analysis layer calls the target test algorithm to perform data analysis on the test data, and obtains the following data analysis results:

[0159] The target test algorithm is called to perform data analysis on the first test data and the second test data to obtain a data analysis result.

[0160] The first test object is a test object belonging to the target test type, the second test object is another test object belonging to the target test type, and the first test object may be different from the second test object. The first test data may be provided by the test user for the first test object, and the second test data may be provided by the test user for the second test object.

[0161] The first test data is test result data generated when the first test object is tested, and the second test data is test result data generated when the second test object is tested.

[0162] The data analysis layer calls the target test algorithm to perform data analysis on the first test data and the second test data to obtain the data analysis result, which may specifically include: calling the target test algorithm to perform data analysis on the first test data to obtain the first analysis result; calling the target test algorithm to perform data analysis on the second test data to obtain the second analysis result. Afterwards, the data analysis result consisting of the first analysis result and the second analysis result may be determined.

[0163] The target test algorithm can perform data analysis on the first test data and the second test data respectively to obtain respective corresponding data analysis results. The analysis process of the first test data and the analysis process of the second test data can be independent of each other.

[0164] The decision analysis layer performs decision analysis on the data analysis results, and obtaining the decision analysis results may specifically include: performing decision analysis on the first analysis result and the second analysis result to obtain a decision analysis result. The decision analysis layer may also determine the optimal analysis result of the first analysis result and the second analysis result through the decision analysis result, and output the optimal analysis result to the test user. The test user may make decisions on the first test object and the second test object through the optimal analysis result to obtain an accurate decision result.

[0165] In one possible design, target test types include: algorithm type.

[0166] The data loading layer determines the first test object and the second test object corresponding to the target test type specifically including:

[0167] Determine a first test algorithm and a second test algorithm corresponding to the algorithm type;

[0168] The first test data provided by the data loading layer loading test user for the first test object specifically includes:

[0169] Loading the first test data provided by the test user for the first test algorithm;

[0170] The second test data provided by the data loading layer loading test user for the second test object specifically includes:

[0171] The second test data provided by the test user for the second test algorithm is loaded.

[0172] Testing the algorithm is an algorithm testing method provided in the embodiment of the present application. The first test algorithm and the second test algorithm can be tested and analyzed to compare the use effects of the two tested algorithms, so as to obtain a better use algorithm through the analysis results and obtain a better quality algorithm. By providing a data analysis system to test the use of the two algorithms, the algorithm testing efficiency can be improved and a more accurate algorithm can be obtained.

[0173] When testing the first test algorithm and the second test algorithm, the actual use results of the two test algorithms can be tested, that is, the use results of the two algorithms can be obtained to perform algorithm testing. Taking the test algorithm as a click-through rate prediction algorithm as an example, in some embodiments, the data loading layer determines that the first test algorithm and the second test algorithm corresponding to the algorithm type may specifically include:

[0174] Determine a first click rate prediction algorithm and a second click rate prediction algorithm corresponding to the algorithm type;

[0175] The first test data provided by the data loading layer loading test user for the first test algorithm may specifically include:

[0176] Loading the first prediction result data provided by the test user for the first click rate prediction algorithm;

[0177] The second test data provided by the data loading layer loading test user for the second test algorithm may specifically include:

[0178] Loading the second prediction result data provided by the test user for the second click rate prediction algorithm;

[0179] The data analysis layer calls the target test algorithm to perform data analysis on the first test data and the second test data, and obtains the data analysis results, which specifically include:

[0180] The target test algorithm is called to perform data analysis on the first prediction result data and the second prediction result data to obtain a data analysis result.

[0181] As a possible implementation, the first prediction result data includes: a plurality of first click-through rates; the plurality of first click-through rates are respectively prediction results obtained by performing click-through rate prediction on a plurality of hot words by a first click-through rate prediction algorithm;

[0182] The second prediction result data includes: a plurality of second click-through rate prediction results; the plurality of second click-through rate prediction results are respectively prediction results of click-through rate prediction of a plurality of hot words by a second click-through rate prediction algorithm.

[0183] like Figure 3 As shown, it is a schematic diagram of the structure of an embodiment of a data system provided in an embodiment of the present application. The development system may include: a development subsystem 31, a test subsystem 32 and a data processing subsystem 33.

[0184] The development subsystem 31 may be used to: detect a target test type selected by a development user from at least one candidate test type; provide the target test type to a data processing system; and generate a data test system corresponding to the test subsystem 32 and the data processing subsystem 33 .

[0185] The test subsystem 32 may be used to: display the target test type to the test user; obtain the test data provided by the test user according to the target test type; and provide the test data to the data processing system.

[0186] The data processing subsystem 33 may include: a data loading layer, used to determine a target test type from at least one candidate test type; load the test data to be tested provided by the test user according to the target test type; load the configuration information corresponding to the test system; a data processing layer, used to parse the configuration information to obtain a parsing result; based on the parsing result, select a target test algorithm from at least one test algorithm; a data analysis layer, used to call the target test algorithm, perform data analysis on the test data, and obtain a data analysis result.

[0187] Among them, the data processing subsystem can correspond to Figure 1 The data processing system in the embodiment shown. The development subsystem may correspond to Figure 1 The development system involved in the embodiment shown. The test subsystem may correspond to Figure 1 The test system in the embodiment shown. The data test system corresponds to Figure 4 A data testing system according to the illustrated embodiment.

[0188] In some embodiments, the data loading layer determining the target test type from at least one candidate test type may specifically include: detecting a test environment of the test system; determining a target test type that matches the test environment from at least one candidate test type;

[0189] The data to be tested provided by the data loading layer loading test user according to the target test type may include:

[0190] Load test data provided by the user in the test system according to the target test type.

[0191] The configuration information corresponding to the data loading layer loading test system may specifically include:

[0192] Load test users based on configuration information provided by the test system.

[0193] As an embodiment, the development subsystem can also be used to: detect the test environment selected by the development user for the test system. The data loading layer can also be used to:

[0194] Determine the test environment and the target test type of the test system based on the test environment selected by the development user in the development system and the target test type selected from at least one candidate test type;

[0195] The data loading layer can also be used to:

[0196] Establish an association between the test environment and the target test type for the test system.

[0197] like Figure 4 As shown, it is a structural schematic diagram of an embodiment of a data testing system provided in an embodiment of the present application. The data testing system may include: a testing subsystem 41 and a data processing subsystem 42.

[0198] The test subsystem 41 can be used to: display the target test type to the test user; obtain the test data provided by the test user according to the target test type; and provide the test data to the data processing subsystem.

[0199] The data processing subsystem 42 can be used for: a data loading layer, which is used to determine a target test type from at least one candidate test type; to load the test data provided by the test user according to the target test type; to load the configuration information corresponding to the test subsystem 41; a data processing layer, which is used to parse the configuration information and obtain a parsing result; based on the parsing result, select a target test algorithm from at least one test algorithm; a data analysis layer, which is used to call the target test algorithm, perform data analysis on the test data, and obtain a data analysis result.

[0200] Among them, the data processing subsystem 42 may correspond to Figure 1 The data processing system of the embodiment shown. The test subsystem may correspond to Figure 1 The test system involved in the embodiment shown. The data test system can be composed of Figure 3 The development system shown is generated.

[0201] In some embodiments, the data loading layer determining the target test type from at least one candidate test type may specifically include: detecting a test environment of the test system; determining a target test type that matches the test environment from at least one candidate test type;

[0202] The data to be tested provided by the data loading layer loading test user according to the target test type may include:

[0203] Load test data provided by the user in the test system according to the target test type.

[0204] The configuration information corresponding to the data loading layer loading test system may specifically include:

[0205] Load test users based on configuration information provided by the test system.

[0206] As an embodiment, the development subsystem can also be used to: detect the test environment selected by the development user for the test system. The data loading layer can also be used to:

[0207] Determine the test environment and the target test type of the test system based on the test environment selected by the development user in the development system and the target test type selected from at least one candidate test type;

[0208] The data loading layer can also be used to:

[0209] Establish an association between the test environment and the target test type for the test system.

[0210] The test subsystem may also be used to: display the test environment and the target test type to the test user so that the test user can learn about the test environment and the target test type.

[0211] The following will take the A / B testing scenario in the Internet field as an example to explain the technical solution of the present application in detail.

[0212] Figure 5a This is a configuration example diagram of a data testing system 50 provided in an embodiment of the present application.

[0213] refer to Figure 5a The data processing subsystem 501 can be a test engine in the field of Internet A / B testing, and can form a data testing system 50 with the testing subsystem 502 and be configured in a testing device M1.

[0214] The test subsystem 502 may be a test platform for implementing test functions based on the data processing subsystem 501. Test users may input test data and configuration information into the test subsystem 502. The data loading layer in the data processing subsystem 501 may load the test data and configuration information. The data processing layer may parse the configuration information to obtain a parsing result, and select a target test algorithm from at least one test algorithm based on the parsing result. Afterwards, the data analysis layer may call the target test algorithm to perform data analysis on the test data to obtain a data analysis result.

[0215] exist Figure 5a In the illustrated embodiment, the data processing subsystem 501 and the test subsystem 502 are both configured in the test device M1. However, in some embodiments, the data processing subsystem 501 and the test subsystem 502 can also be configured in different devices, for example, the data processing subsystem 501 can be configured in a cloud server, and the test subsystem 502 can be configured in a user's notebook, tablet computer or other electronic device. The specific configuration of the data processing subsystem 501 and the test subsystem 502 in this embodiment is only exemplary, and the configuration device can be changed according to actual use needs.

[0216] Figure 5b This is a configuration example diagram of a data system 51 provided in an embodiment of the present application.

[0217] The development subsystem 511 can be a platform for developing the test environment and target test type of the test subsystem 513 based on the data processing subsystem 512. The data system 51 formed by the development subsystem 511, the data processing subsystem 512 and the test subsystem 513 can be configured in a development device M2. Therefore, the development user can set the data processing subsystem 512 and the test subsystem 513 through the development subsystem 511 in the development device M2, thereby obtaining a data test system.

[0218] like Figure 6 FIG. 1 is a flowchart of an embodiment of a data processing method provided by an embodiment of the present application. The method may include:

[0219] Step 601: Determine a target test type from at least one candidate test type.

[0220] Step 602: Load the test data provided by the test user according to the target test type.

[0221] Step 603: Load the configuration information corresponding to the test system.

[0222] Step 604: parse the configuration information to obtain a parsing result.

[0223] Step 605: Based on the analysis result, select a target test algorithm from at least one test algorithm.

[0224] Step 606: Call the target test algorithm, perform data analysis on the test data, and obtain data analysis results.

[0225] In the embodiment of the present application, the target test type can be determined from at least one candidate test type. By providing at least one candidate test type, the test types that the test system can support can be increased, and the actual test scope can be expanded. The load test user completes the loading of the data to be tested and the configuration information corresponding to the load test system according to the test data provided by the target test type and the configuration information corresponding to the load test system, realizes modular data processing, and improves data processing efficiency. The configuration information is parsed and processed to obtain the parsing result, and then based on the parsing result, the target test algorithm is selected from at least one test algorithm. By supporting the selection of at least one test algorithm, the test system can support more test methods and expand the application scope of the test system. Then call the target test algorithm to perform data analysis on the test data, obtain the data analysis result, and complete the test of the test system. By increasing the supported test types and expanding the supported test methods, the use function of the test engine is expanded, thereby improving the use scope of the test system, so that the test system supports testing of multiple test types and multiple test methods, and improves the utilization efficiency of the test system.

[0226] In an optional embodiment, the method further comprises:

[0227] Perform decision analysis on the data analysis results to obtain decision analysis results.

[0228] In some embodiments, determining the target test type from at least one candidate test type may include:

[0229] Detect the test environment of the test system;

[0230] Determining a target test type that matches the test environment from at least one candidate test type;

[0231] The test data provided by the load test user according to the target test type may include:

[0232] Load test data provided by the user in the test system according to the target test type.

[0233] The configuration information corresponding to the load test system may include:

[0234] Load test users based on configuration information provided by the test system.

[0235] In an optional embodiment, the method further comprises:

[0236] Output the decision analysis results through the decision output interface.

[0237] In an optional embodiment, the load test user provides the test data according to the target test type; the configuration information corresponding to the load test system may include:

[0238] Loading the test data provided by the test user according to the target test type through the data loading interface;

[0239] The configuration information corresponding to the test system is loaded through the data loading interface.

[0240] In an optional embodiment, determining the target test type from at least one candidate test type may include: determining the target test type associated with the test environment from at least one candidate test type based on the test environment of the current test system.

[0241] In an optional embodiment, the method may further include:

[0242] The test environment and the target test type of the test system are determined according to the test environment selected by the development user in the development system and the target test type selected from at least one candidate test type.

[0243] Optionally, the method may further include: establishing an association relationship between the test environment and the target test type for the test system.

[0244] In an optional embodiment, the data to be tested provided by the loading test user according to the target test type and the configuration information provided by the test user include:

[0245] The load test user provides the test data in the test system according to the target test type and the configuration information provided by the test user in the test system.

[0246] In an optional embodiment, calling a target test algorithm, performing data analysis on the test data, and obtaining the data analysis results include:

[0247] The target test algorithm is called through the algorithm calling interface, and data analysis and processing are performed on the test data to obtain data analysis results.

[0248] In an optional embodiment, the method may further include:

[0249] At least one test algorithm determined by a development user in a development system is obtained.

[0250] In an optional embodiment, the method may further include:

[0251] Obtain the first processing algorithm of the custom input by the development user in the development system;

[0252] Acquire a second processing algorithm selected by a development user from at least one candidate processing algorithm provided by the data processing system;

[0253] At least one test algorithm consisting of the first processing algorithm and the second processing algorithm is determined.

[0254] In an optional embodiment, the method may further include:

[0255] Performing data aggregation processing on the data to be tested to obtain first data;

[0256] converting the first data into second data corresponding to a standard data format predefined by the data processing system;

[0257] Call the target test algorithm to perform data analysis on the test data. The data analysis results may include:

[0258] The target test algorithm is called to perform data analysis on the second data to obtain a data analysis result.

[0259] As an embodiment, the method may further include:

[0260] The data analysis results are output to the test system so that the test system can output the data analysis results to the test user so that the test user can confirm the data analysis results.

[0261] As another embodiment, the to-be-tested data provided by the load test user according to the target test type may include:

[0262] determining at least one test object corresponding to a target test type;

[0263] The test data to be tested provided by the test user for at least one test object is loaded.

[0264] In some embodiments, determining at least one test object corresponding to the target test type may include:

[0265] Determining a first test object and a second test object corresponding to a target test type;

[0266] The data to be tested provided by the load test user for at least one test object may include:

[0267] Loading first test data provided by a test user for a first test object;

[0268] Loading second test data provided by the test user for the second test object;

[0269] Call the target test algorithm to perform data analysis on the test data. The data analysis results may include:

[0270] The target test algorithm is called to perform data analysis on the first test data and the second test data to obtain a data analysis result.

[0271] In one possible design, the target test types include: algorithm type;

[0272] Determining a first test object and a second test object corresponding to a target test type may include determining a first test algorithm and a second test algorithm corresponding to an algorithm type.

[0273] The first test data provided by the load test user for the first test object may include:

[0274] Loading the first test data provided by the test user for the first test algorithm;

[0275] The second test data provided by the load test user for the second test object may include:

[0276] The second test data provided by the test user for the second test algorithm is loaded.

[0277] In an optional embodiment, determining a first test algorithm and a second test algorithm corresponding to the algorithm type may include: determining a first click rate prediction algorithm and a second click rate prediction algorithm corresponding to the algorithm type.

[0278] The first test data provided by the load test user for the first test algorithm may include:

[0279] Loading the first prediction result data provided by the test user for the first click rate prediction algorithm;

[0280] The second test data provided by the load test user for the second test algorithm may include:

[0281] Loading the second prediction result data provided by the test user for the second click rate prediction algorithm;

[0282] Calling the target test algorithm, performing data analysis on the first test data and the second test data, and obtaining the data analysis results may include:

[0283] The target test algorithm is called to perform data analysis on the first prediction result data and the second prediction result data to obtain a data analysis result.

[0284] Further, optionally, the first prediction result data includes: a plurality of first click rates; the plurality of first click rates are respectively prediction results obtained by performing click rate prediction on a plurality of hot words by a first click rate prediction algorithm;

[0285] The second prediction result data includes: a plurality of second click-through rate prediction results; the plurality of second click-through rate prediction results are respectively prediction results of click-through rate prediction of a plurality of hot words by a second click-through rate prediction algorithm.

[0286] It should be noted that the specific implementation method and technical effect of the data processing method provided in the embodiment of the present application are the same as the processing steps in the data processing system in the aforementioned embodiment. For the sake of brevity, they will not be repeated here.

[0287] The execution subject of each step of the data processing method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 601 to 605 can be device A; for another example, the execution subject of steps 601 and 604 can be device A, and the execution subject of step 605 can be device B; and so on.

[0288] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel, and the sequence numbers of the operations, such as 601, 602, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.

[0289] An embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the data processing method in the aforementioned embodiment.

[0290] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0291] like Figure 7 FIG. 1 is a flowchart of another embodiment of a data processing method provided in an embodiment of the present application. The method may include:

[0292] 701: Display the target test type to the test user.

[0293] 702: Obtain the test data provided by the test user according to the target test type.

[0294] 703: Provide the test data to the data processing system.

[0295] The specific implementation method and technical effect of the data processing method provided in the embodiment of the present application are the same as the processing steps in the test system in the aforementioned embodiment, and for the sake of brevity, they are not repeated here. In addition, the interaction process with the data processing system can also refer to the description in the aforementioned embodiment, and they are not repeated here.

[0296] like Figure 8 FIG. 1 is a flowchart of another embodiment of a data processing method provided in an embodiment of the present application. The method may include:

[0297] 801: Detect a target test type selected by a development user from at least one candidate test type.

[0298] 802: Provide a target test type to a data processing system.

[0299] 803: Generate a test system and a data test system corresponding to the data processing system.

[0300] The specific implementation method and technical effect of the data processing method provided in the embodiment of the present application are the same as the processing steps in the development system in the aforementioned embodiment, and are not repeated here for the sake of brevity. In addition, the interaction process with the data processing system and the test system can also refer to the description in the aforementioned embodiment, and is not repeated here.

[0301] like Fig. 9 As shown, Figure 1~Figure 2 The data processing system shown may be configured in a computing device, which may include: a storage component 901 and a processing component 902 .

[0302] The storage component 901 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 902 to implement Figure 6 The data processing method shown.

[0303] The processing component 902 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0304] The storage component 901 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0305] Of course, the computing device may also include other components, such as input / output interfaces, communication components, etc. The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices.

[0306] In addition, if Fig.10 As shown, Figure 3 The data system shown can be implemented as an electronic device, which can be, for example, Figure 5b The development device shown in the figure may include: a storage component 1001 and a processing component 1002.

[0307] The storage component 1001 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 1002 to implement Figure 6 , Figure 7 as well as Figure 8 The data processing method shown.

[0308] The processing component 1002 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above data processing method.

[0309] The storage component 1001 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0310] Of course, the electronic device may also include other components, such as input / output interfaces, communication components, etc. The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc. The communication component is configured to facilitate wired or wireless communication between the electronic device and other devices.

[0311] like Fig.11 As shown, Figure 4 The data test system shown can be implemented as an electronic device, which can be Figure 5a The test device shown may include: a storage component 1101 and a processing component 1102 .

[0312] The storage component 1101 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 1102 to implement Figure 6 as well as Figure 7 The data processing method shown.

[0313] The processing component 1102 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above data processing method.

[0314] The storage component 1101 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0315] Of course, the electronic device may also include other components, such as input / output interfaces, communication components, etc. The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc. The communication component is configured to facilitate wired or wireless communication between the electronic device and other devices.

[0316] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0317] Computer program instructions in an electronic device or computing device may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0318] The computer program instructions in the electronic device or computing device may also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0319] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing system, characterized in that: include: A data loading layer is used to detect the test environment of the test system, and based on the association between the test environment and the target test type, determine a target test type that matches the test environment from at least one candidate test type, wherein the at least one candidate test type includes a streaming test type and a batch test type; determine at least one test object corresponding to the target test type; load the test data provided by the test user for the at least one test object through a data loading interface; load the configuration information corresponding to the test system through the data loading interface, wherein different test types correspond to different data loading interfaces; The data processing layer is used to parse the configuration information and obtain the parsing result; Based on the analysis result, selecting a target test algorithm from at least one test algorithm; Performing data aggregation processing on the data to be tested to obtain first data; converting the first data into second data corresponding to a standard data format predefined by the data processing system; A data analysis layer, used to call the target test algorithm, perform data analysis on the second data, and obtain data analysis results; The decision analysis layer is used to perform multiple decision analysis processes on the data analysis results using multiple decision sub-algorithms to obtain multiple decision result data, and comprehensively confirm whether there are significant differences between the test data corresponding to each of the tested objects based on at least one decision result data among the multiple decision result data.

2. The system according to claim 1, characterized in that The data loading layer is also used to: Based on the test environment selected by the development user in the development system and the target test type selected from the at least one candidate test type, the test environment and the target test type of the test system are determined.

3. The system according to claim 1, characterized in that The data analysis layer is also used to: Obtain the first processing algorithm of the custom input by the development user in the development system; Acquire a second processing algorithm selected by the development user from at least one candidate processing algorithm provided by the data processing system; The at least one test algorithm composed of the first processing algorithm and the second processing algorithm is determined.

4. The system according to claim 1, characterized in that Also includes: The data output layer is used to output the data analysis result to the test system, so that the test system can output the data analysis result to the test user, so that the test user can confirm the data analysis result.

5. The system according to claim 1, characterized in that The data loading layer determines at least one test object corresponding to the target test type specifically including: Determining a first test object and a second test object corresponding to the target test type; The data loading layer loading the test data provided by the test user for the at least one test object specifically includes: Loading first test data provided by the test user for the first test object; Loading second test data provided by the test user for the second test object; The data analysis layer calls the target test algorithm to perform data analysis on the data to be tested, and obtaining the data analysis result specifically includes: The target test algorithm is called to perform data analysis on the first test data and the second test data to obtain a data analysis result.

6. The system according to claim 5, characterized in that The target test types include: algorithm type; The data loading layer determines the first test object and the second test object corresponding to the target test type specifically including: Determine a first test algorithm and a second test algorithm corresponding to the algorithm type; The data loading layer loads the first test data provided by the test user for the first test object specifically including: Loading first test data provided by the test user for the first test algorithm; The data loading layer loads the second test data provided by the test user for the second test object specifically including: The second test data provided by the test user for the second test algorithm is loaded.

7. A data testing system, characterized in that: include: Testing subsystem and data processing subsystem; The test subsystem is used to: display the target test type to the test user; Acquire the test data provided by the test user according to the target test type; Providing the data to be tested to the data processing subsystem; The data processing subsystem includes: a data loading layer, which is used to detect the test environment of the test system, and based on the association between the test environment and the target test type, determine the target test type that matches the test environment from at least one candidate test type, and at least one candidate test type includes a streaming test type and a batch test type; determine at least one test object corresponding to the target test type; load the test data provided by the test user for the at least one test object through a data loading interface; load the configuration information corresponding to the test subsystem through the data loading interface, and different test types correspond to different data loading interfaces; the data processing layer is used to parse and process the configuration information to obtain Analysis results; based on the analysis results, select a target test algorithm from at least one test algorithm, perform data aggregation processing on the data to be tested to obtain first data; convert the first data into second data corresponding to the standard data format predefined by the data processing subsystem; a data analysis layer, used to call the target test algorithm, perform data analysis processing on the second data, and obtain data analysis results; a decision analysis layer, used to use multiple decision sub-algorithms to perform multiple decision analysis processing on the data analysis results, obtain multiple decision result data, and comprehensively confirm whether there are significant differences between the test data corresponding to each of the tested objects based on at least one decision result data among the multiple decision result data.

8. A data system, characterized in that: include: Development subsystem, test subsystem and data processing subsystem; The development subsystem is used to: detect a target test type selected by a development user from at least one candidate test type; Providing the target test type to the test subsystem; generating a data test system corresponding to the test subsystem and the data processing subsystem; The test subsystem is used to: display the target test type to the test user; Acquire the test data provided by the test user according to the target test type; Providing the data to be tested to the data processing subsystem; The data processing subsystem includes: a data loading layer, which is used to detect the test environment of the test system, and based on the association between the test environment and the target test type, determine the target test type that matches the test environment from at least one candidate test type, and at least one candidate test type includes a streaming test type and a batch test type; determine at least one test object corresponding to the target test type; load the test data provided by the test user for the at least one test object through a data loading interface; load the configuration information corresponding to the test subsystem through the data loading interface, and different test types correspond to different data loading interfaces; the data processing layer is used to parse and process the configuration information to obtain Analysis results; based on the analysis results, select a target test algorithm from at least one test algorithm, perform data aggregation processing on the data to be tested to obtain first data; convert the first data into second data corresponding to the standard data format predefined by the data processing subsystem; a data analysis layer, used to call the target test algorithm, perform data analysis processing on the second data, and obtain data analysis results; a decision analysis layer, used to use multiple decision sub-algorithms to perform multiple decision analysis processing on the data analysis results, obtain multiple decision result data, and comprehensively confirm whether there are significant differences between the test data corresponding to each of the tested objects based on at least one decision result data among the multiple decision result data.

9. A data processing method, characterized in that: include: Detecting a test environment of the test system, and determining a target test type that matches the test environment from at least one candidate test type based on an association relationship between the test environment and the target test type, wherein the at least one candidate test type includes a stream test type and a batch test type; Determine at least one test object corresponding to the target test type; load the to-be-tested data provided by the test user for the at least one test object through a data loading interface; Load the configuration information corresponding to the test system through the data loading interface, and different test types correspond to different data loading interfaces; Parsing the configuration information to obtain a parsing result; Based on the analysis result, selecting a target test algorithm from the at least one test algorithm; Performing data aggregation processing on the data to be tested to obtain first data; converting the first data into second data corresponding to a standard data format predefined by a data processing system; Calling the target test algorithm, performing data analysis processing on the second data, and obtaining a data analysis result; The data analysis results are processed multiple times by using multiple decision sub-algorithms to obtain multiple decision result data, and whether there is a significant difference between the test data corresponding to each of the tested objects is comprehensively confirmed based on at least one decision result data among the multiple decision result data.

Images