Data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data processing method and device, electronic equipment and a storage medium, and relates to the technical field of data processing. A specific embodiment of the method comprises the following steps: receiving an experiment instruction, obtaining an experiment demand in the experiment instruction, disassembling the experiment demand, and determining a parameter value set corresponding to each preset experiment parameter; based on the parameter value set, matching a corresponding service data table and an experiment shunting table, and associating the service data table with the experiment shunting table to generate a data detail table; extracting an aggregation dimension from the parameter value set to call a preset aggregation program, and executing data aggregation on the data detail list to obtain experimental data; responding to the condition that the experiment task is triggered, obtaining corresponding experiment data and executing the experiment program to obtain an experiment result. According to the embodiment of the invention, the problem that the AB experiment system builds related experiment data calculation links and related programs through the experience of developers, and the efficiency is low can be solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of mobile Internet, in order to achieve more accurate and refined operations and digital decision-making, data analysis is usually required for verification. For example, AB experiments are used to observe the differences in indicators to achieve the effect of data science quantification. In related technologies, the AB experiment system uses the experience of developers to build relevant experimental data calculation links and related programs to achieve the calculation of relevant experimental indicators, but this method is inefficient. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a data processing method, device, electronic device and storage medium, which can solve the problem of low efficiency in building related experimental data calculation links and related programs through the experience of developers in the AB experiment system.

[0004] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for data processing is provided.

[0005] A data processing method according to an embodiment of the present invention includes: receiving an experimental instruction, obtaining an experimental requirement in the experimental instruction, decomposing the experimental requirement, and determining a parameter value set corresponding to each preset experimental parameter;

[0006] Matching the corresponding business data table and the experimental diversion table based on the parameter value set, and associating the business data table with the experimental diversion table to generate a data detail table;

[0007] Extracting an aggregation dimension from the parameter value set to call a preset aggregation program, perform data aggregation on the data detail table, and obtain experimental data;

[0008] In response to the experimental task being triggered, corresponding experimental data is acquired and the experimental program is executed in combination with the parameter value set to obtain the experimental result.

[0009] In one embodiment, extracting the aggregation dimension from the parameter value set includes:

[0010] An aggregation target is determined based on the parameter value of the experimental indicator in the experimental parameters, and each data field associated with the aggregation target is obtained from the data detail table to extract an aggregation dimension from each of the data fields.

[0011] In yet another embodiment, before calling the preset aggregation program, the method further includes:

[0012] The diversion dimension corresponding to the experimental diversion table is obtained to update the aggregation dimension based on the diversion dimension.

[0013] In yet another embodiment, calling a preset aggregation program to perform data aggregation on the data detail table includes:

[0014] Determine the correlation between each of the aggregation dimensions and the experimental indicators in the experimental parameters to sort the aggregation dimensions, and call the preset aggregation programs in sequence according to the arrangement order to perform data aggregation on the data detail table.

[0015] In yet another embodiment, the experimental requirement is decomposed to determine a parameter value set corresponding to each preset experimental parameter, including:

[0016] Convert the experimental requirements into text information, decompose the text information based on preset experimental parameters, determine parameter values ​​corresponding to the experimental parameters, and obtain a parameter value set;

[0017] Based on each of the parameter values, the composite parameter value of the corresponding experimental parameter is queried to update the parameter value set with the composite parameter.

[0018] In yet another embodiment, calling a preset aggregation program to perform data aggregation on the data list to obtain experimental data includes:

[0019] Query historical aggregation records based on the aggregation dimension and the data detail table;

[0020] In response to the query result being a successful query, the corresponding historical record result is determined as the experimental data.

[0021] To achieve the above objective, according to another aspect of an embodiment of the present invention, a data processing device is provided.

[0022] A data processing device according to an embodiment of the present invention includes: a receiving unit, configured to receive an experimental instruction, obtain an experimental requirement in the experimental instruction, decompose the experimental requirement, and determine a parameter value set corresponding to each preset experimental parameter;

[0023] A generating unit, configured to match a corresponding business data table and an experimental flow distribution table based on the parameter value set, and associate the business data table with the experimental flow distribution table to generate a data detail table;

[0024] An aggregation unit, used for extracting an aggregation dimension from the parameter value set to call a preset aggregation program, perform data aggregation on the data detail table, and obtain experimental data;

[0025] The processing unit is used for acquiring corresponding experimental data in response to the experimental task being triggered and executing the experimental program in combination with the parameter value set to obtain the experimental result.

[0026] In one embodiment, the polymerization unit is specifically used for:

[0027] An aggregation target is determined based on the parameter value of the experimental indicator in the experimental parameters, and each data field associated with the aggregation target is obtained from the data detail table to extract an aggregation dimension from each of the data fields.

[0028] In yet another embodiment, the polymerization unit is specifically used for:

[0029] The diversion dimension corresponding to the experimental diversion table is obtained to update the aggregation dimension based on the diversion dimension.

[0030] In yet another embodiment, the polymerization unit is specifically used for:

[0031] Determine the correlation between each of the aggregation dimensions and the experimental indicators in the experimental parameters to sort the aggregation dimensions, and call the preset aggregation programs in sequence according to the arrangement order to perform data aggregation on the data detail table.

[0032] In yet another embodiment, the polymerization unit is specifically used for:

[0033] Query historical aggregation records based on the aggregation dimension and the data detail table;

[0034] In response to the query result being a successful query, the corresponding historical record result is determined as the experimental data.

[0035] In yet another embodiment, the receiving unit is specifically configured to:

[0036] Convert the experimental requirements into text information, decompose the text information based on preset experimental parameters, determine parameter values ​​corresponding to the experimental parameters, and obtain a parameter value set;

[0037] Based on each of the parameter values, the composite parameter value of the corresponding experimental parameter is queried to update the parameter value set with the composite parameter.

[0038] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.

[0039] An electronic device according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method provided by the embodiment of the present invention.

[0040] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer readable medium is provided.

[0041] A computer-readable medium according to an embodiment of the present invention stores a computer program, and when the program is executed by a processor, the data processing method provided by the embodiment of the present invention is implemented.

[0042] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer program product is provided.

[0043] A computer program product according to an embodiment of the present invention includes a computer program, and when the program is executed by a processor, the data processing method provided by the embodiment of the present invention is implemented.

[0044] One embodiment of the above invention has the following advantages or beneficial effects:

[0045] In an embodiment of the present invention, after receiving an experimental instruction, the experimental requirements in the experimental instruction are obtained, the experimental requirements are disassembled, and the parameter value set corresponding to each preset experimental parameter is determined; then the corresponding business data table and the experimental diversion table can be matched based on the parameter value set to associate the business data table and the experimental diversion table, and the aggregation dimension can be extracted from the parameter value set to call the preset aggregation program to perform data aggregation on the data detail table to obtain the experimental data; so after the experimental task is triggered, the corresponding experimental data can be obtained and the experimental program can be executed to obtain the experimental results. In this way, in an embodiment of the present invention, the detailed data required for the corresponding experiment can be determined according to the experimental requirements and these data can be aggregated in advance, so that the experimental results can be obtained directly through the experimental data during the experiment, and the experimental data calculation link can be determined and the experimental program can be executed according to the experimental requirements, so that the experimental results can meet the experimental requirements without using the experience of the developers, thereby improving the accuracy and efficiency of the experiment.

[0046] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0048] Figure 1 is a schematic diagram of a main flow of a data processing method according to an embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of another main flow of a data processing method according to an embodiment of the present invention;

[0050] Figure 3is a schematic diagram of main units of a data processing apparatus according to an embodiment of the present invention;

[0051] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0052] Figure 5 It is a schematic diagram of the structure of a computer system suitable for implementing the embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0054] It should be pointed out that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict. The acquisition, transmission, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations. In the embodiments of this application, some existing solutions in the industry such as certain software, components, and models may be mentioned, which should be considered as exemplary. Their purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0055] An embodiment of the present invention provides an experimental parameter system, which can be used in a scenario of experimental data processing, and specifically can be used in a scenario of AB experimental data processing.

[0056] The embodiment of the present invention provides a data processing method, which can be executed by a data processing system. Figure 1 As shown, the method includes the following steps.

[0057] S101: receiving an experiment instruction, obtaining an experiment requirement in the experiment instruction, analyzing the experiment requirement, and determining a parameter value set corresponding to each preset experiment parameter.

[0058] Among them, the experimental instructions can be sent to the data processing system in a variety of ways, such as message sending, voice input, information collection, etc. The experimental requirement instruction can represent the requirements of this experiment. After receiving the experimental instruction, the experimental requirements can be obtained from it to process it. In this step, a disassembly model for disassembling the experimental requirements can be preset to disassemble the parameter value set of each preset experimental parameter from the experimental requirements.

[0059] Specifically, in an embodiment of the present invention, the experimental requirements may be specifically text information, so the disassembly model may be a text recognition model, which performs text recognition on the experimental requirements to disassemble the parameter values ​​of each experimental parameter therefrom, thereby obtaining a parameter value set.

[0060] In an embodiment of the present invention, the experimental parameters may be pre-set, which are usually used to represent the parameters involved in data processing during the experiment. For example, the experimental parameters may include: experimental indicators, calculation logic, modification conditions, and atomic information. The experimental indicators represent the indicators to be observed in this experiment, the atomic information represents the fields required to calculate the experimental indicators, the calculation logic represents the calculation logic required to obtain the experimental results, and the modification conditions represent the conditions required to limit the experimental indicator values ​​and calculations.

[0061] For example, the experimental requirement information can be: see the year-on-year growth of the average transaction amount contributed by users in a certain brand of retail products in a certain period of time. From this, we can see that the experimental indicator is the average user contribution amount, the calculation logic is year-on-year, the atomic information includes retail, transaction, order, number of users, transaction amount, and the modification condition is a certain brand.

[0062] It should be noted that, when performing the experimental parameter decomposition, further decomposition can be performed according to the specific content of the experimental indicators to facilitate subsequent data calculations. For example, the experimental indicator is the average contribution amount of users, and its calculation method is usually obtained by dividing the transaction amount by the number of transaction users. Therefore, when decomposing the experimental indicator, it can be decomposed into composite indicators (composite parameter values), namely the transaction amount and the number of transaction users. The corresponding atomic information can be determined separately according to this composite indicator. Modification conditions can be divided into general conditions and special conditions according to needs. General conditions are conditions applicable to all experimental indicators, and special conditions can be conditions applicable to some experimental indicators. And each experimental parameter can be decomposed according to the specific experimental requirements. When there is no parameter value corresponding to the experimental parameter, the parameter value can be empty.

[0063] In an embodiment of the present invention, the decomposition of the experimental requirements can realize the conversion of the business language into the corresponding indicator language, that is, the information required for the business experiment is converted into the parameters required in the experiment, so as to facilitate the logical conversion between the experimental objectives and indicators. In an embodiment of the present invention, the model for decomposition can be pre-set and trained, such as using algorithms such as machine learning or neural networks.

[0064] Specifically, in the embodiment of the present invention, the associations of the experimental parameters can be pre-sorted so that the parameter values ​​can be disassembled through the pre-sorted associations when disassembling. For example, the internal relationship between indicators can be pre-determined: a) non-general statistical algorithm relationship, such as derived indicators, which can be maintained to the data table clipping and can be described in natural language; b) general dynamic relationship: this relationship can be maintained to user indicator alarms, monitoring rules, etc.; c) fact-based mathematical four static relationships: this relationship can be maintained to the determination of composite indicators. The calculation logic between indicators can also be pre-determined: such as time same-year-on-year, recent X-day proportion, etc., the complex calculation logic can be maintained to the function. The indicator modification conditions can also be pre-determined: if multi-level modifications can be split, the modification content can be maintained to multi-level indicators; if there is modification information with a high degree of reuse, the modification content can be maintained to the data table clipping, and structured information description (dimension, operator, dimension value) can be used. The atomic information of the indicator can also be pre-determined: the remaining business language and related business indicators are disassembled, such as business system / business domain, theme, subject, business process, measurement, etc. In the embodiment of the present invention, corresponding modification conditions can also be converted for different business scenarios. For example, for a live broadcast scenario, it can be converted into a preset corresponding modification condition; and the calculation of experimental indicator statistics can be converted into calculation logic, such as converting the sample size, sample sum, sample square sum, etc. into corresponding indicator functions.

[0065] Based on the sorting and pre-setting of the above relationships, the experimental requirements can be broken down and used for subsequent experimental operations.

[0066] S102: Matching the corresponding business data table and the experimental diversion table based on the parameter value set, and associating the business data table with the experimental diversion table to generate a data detail table.

[0067] Among them, after determining the parameter value of each experimental parameter, the corresponding data table can be matched to be used as sample data for the experiment. The business data table represents the business data generated by the business processing corresponding to the experimental parameters, which is usually obtained from the actual business processing, such as a fact table. The experimental diversion table represents the data table generated by data diversion according to the experimental requirements. Usually, experimental diversion is to divide the data into an experimental group and a control group for the purpose of judging and comparing the experimental results. This step associates the business data table with the experimental diversion table to generate a data detail table.

[0068] Specifically, the experimental sample data is usually determined based on the experimental indicators, so in this step, the business data table and the experimental diversion table can be matched according to the parameter values ​​of the experimental indicators in the experimental parameters. For example, if the experimental indicator is the average contribution amount of the user, the matched business data table may include the user transaction order data table, and the experimental diversion table may be divided according to the user, or the corresponding experimental diversion table may be matched.

[0069] In an embodiment of the present invention, an association between a business data table and an experimental diversion table can be established based on a logical table. For example, a logical association can be established by way of RIGHT JOIN to obtain a data detail table. The data detail table is equivalent to a data table that is widened and heightened by the business data table and the experimental diversion table. It supports more flexible experimental data observation while ensuring homogeneity and comparability. At the same time, the ability of logical dimensional expansion is used to avoid the problems of storage and computing redundancy and poor scalability at the physical level.

[0070] It should be noted that in the embodiment of the present invention, the data processing system can also provide a configuration interface so that the user can configure the association between the business data table and the experimental diversion table according to the needs, that is, to establish the association logic between the two. When the association configuration is not entered, the association can be established based on the common dimensions in the business data table and the experimental diversion table, such as the user identification dimension. In the embodiment of the present invention, the business data table and the experimental diversion table can be pre-stored in the database to facilitate query and association.

[0071] S103: extracting an aggregation dimension from the parameter value set to call a preset aggregation program, perform data aggregation on the data detail table, and obtain experimental data.

[0072] Among them, data aggregation means aggregating the data in the data detail table through different dimensions to obtain the experimental data of the dimension required for the experiment. In this step, the aggregation dimension can be determined according to the parameter value set. The aggregation program is pre-set. After determining the aggregation dimension, the preset aggregation program can be called to perform data aggregation on the data detail table from the aggregation dimension to obtain experimental data. Experimental data refers to the data required for the laboratory.

[0073] Specifically, the aggregation dimensions extracted in this step may include one or more, which can be specifically performed as follows: determining the aggregation target based on the parameter value of the experimental indicator in the experimental parameters, obtaining each data field associated with the aggregation target from the data details table, and extracting the aggregation dimension from each data field.

[0074] The aggregation target can be the parameter value of the experimental indicator or the composite indicator corresponding to the experimental indicator. Since the data stored in the data details table includes multiple dimensions, that is, multiple fields, it can be aggregated from each data field associated with the aggregation target, that is, the aggregation dimension can be extracted from each data field.

[0075] For example, the data details table may include the experimental indicator: transaction amount, and also include data fields including: user ID, order ID, item ID, store ID, etc. Therefore, user ID, order ID, item ID, store ID can all be used as aggregation dimensions for data aggregation to calculate transaction amount data.

[0076] It should be noted that in the embodiment of the present invention, aggregation can be performed from the sample granularity aggregation and the experimental granularity. The sample granularity indicates that the aggregation dimension is determined from the business data perspective, and the experimental granularity indicates that the aggregation dimension is determined from the experimental diversion dimension. Therefore, in this step, the diversion dimension corresponding to the experimental diversion table can also be obtained to update it to the aggregation dimension.

[0077] In one embodiment, the step of aggregation in this step can be performed as follows: determine the correlation between each aggregation dimension and the experimental indicator in the experimental parameters to sort the aggregation dimensions, and call the preset aggregation program in sequence according to the arrangement order to perform data aggregation on the data details table.

[0078] The correlation between the aggregation dimension and the experimental indicator in the experimental parameters can be determined in a variety of ways. For example, the dimension that appears in the modification condition corresponding to the experimental indicator is usually highly correlated with the experimental indicator, so the aggregation dimensions with different correlations can be determined, and the data can be aggregated in sequence after the aggregation dimensions are sorted. The preset program can be pre-set, and after determining the aggregation dimension and the data detail table, the aggregation program can be triggered based on this to perform the corresponding aggregation operation.

[0079] It should be noted that in order to simplify the aggregation operation, in the embodiment of the present invention, after determining the aggregation dimension and the data detail table, the historical aggregation record can be queried first to determine whether the current aggregation operation has been performed in the historical aggregation operation, that is, the historical aggregation record is queried based on the aggregation dimension and the data detail table; if the query result is that the query is successful, the corresponding historical record result is determined as the experimental data; if the query result is that the query is unsuccessful, the aggregation operation can be performed. In the embodiment of the present invention, when there are many aggregation dimensions, only part of the aggregation operation can be screened out from the aggregation dimension without executing all of them.

[0080] In the embodiment of the present invention, the idea of ​​Flow-Node can be introduced to build the aggregation operation of each dimension in a single node. In the execution stage, based on the meta-information transmitted by AB and the HBO policy information maintained by the automation system, the skipping and merging calculation operations of related nodes are automatically completed; and combined with the automatic release of operation tasks and automatic biochemical degradation technology, the physical deletion of related data and the disabling of corresponding tasks can be automatically completed within a period of time after the end of the experiment. Among them, in each aggregation operation, two time strategies can be included, namely the real-time time strategy and the cumulative time strategy, which are used to calculate the aggregation results corresponding to different time strategies of the experimental indicators respectively.

[0081] It should be noted that, in the embodiment of the present invention, the data detail processing may also include multiple methods, such as column-to-row processing, etc., to obtain experimental data that meets the experimental requirements.

[0082] S104: In response to the experimental task being triggered, corresponding experimental data is acquired and the experimental program is executed in combination with the parameter value set to obtain an experimental result.

[0083] Among them, the experimental task can be triggered automatically or be triggered. After the experimental task is triggered, the experimental program can be executed based on the corresponding experimental data and parameter value set. In this step, the corresponding experimental process can be constructed according to the parameter value set and the preset experimental program, so that the experimental results can be obtained through the experimental data execution, which can be used for experimental analysis.

[0084] Specifically, in the embodiment of the present invention, the experimental process can be constructed according to the SQL engine, and the abstract calculation logic can be solidified according to the atomic information and modification conditions in the experimental parameters to generate the calculation process of the experimental indicators, and then the calculation logic can be combined to generate the process of calculating the experimental results to obtain the experimental results.

[0085] It should be noted that the embodiments of the present invention can be specifically used for AB experiments. Based on the constructed DAG dynamic scheduling capability, through periodic task triggering, task status management, alarm monitoring, queue concurrency control, etc., the data processing of each node information is combined with other experimental data to complete the automatic construction of the AB experiment process, and it can also be combined with BDP task release and DMS modeling platform to systematically complete experimental task submission and experimental model creation to meet the experimental needs in the embodiments of the present invention. In this way, the automated process can avoid frequent script release, task deployment, model post-governance and other manual operations performed by development personnel in the original R&D operation path, thereby improving the accuracy and efficiency of the experiment.

[0086] In an embodiment of the present invention, the detailed data required for the corresponding experiment can be determined according to the experimental requirements and the data can be aggregated in advance, so that the experimental results can be obtained directly through the experimental data during the experiment, and the experimental data calculation link can be determined and the experimental program can be executed according to the experimental requirements, so that the experimental results can meet the experimental requirements without utilizing the experience of the developers, thereby improving the accuracy and efficiency of the experiment.

[0087] Combine the following Figure 1 The embodiment shown in the figure specifically describes the method for data processing in the embodiment of the present invention. Figure 2 As shown, the method includes the following steps.

[0088] S201: Receive an experiment instruction and obtain the experiment requirements in the experiment instruction.

[0089] S202: Convert the experimental requirements into text information, decompose the text information based on preset experimental parameters, determine parameter values ​​corresponding to the experimental parameters, and obtain a parameter value set.

[0090] S203: Query the composite parameter value of the corresponding experimental parameter based on the parameter value to update the composite parameter to a parameter value set.

[0091] S204: Match the corresponding business data table and the experimental diversion table based on the parameter value set, associate the business data table with the experimental diversion table, and generate a data detail table.

[0092] S205: Determine an aggregation target based on the parameter value of the experimental indicator in the experimental parameters, obtain each data field associated with the aggregation target from the data detail table, and extract an aggregation dimension from each data field.

[0093] S206: Determine the correlation between each aggregation dimension and the experimental indicator to sort the aggregation dimensions, and call the preset aggregation program in sequence according to the arrangement order to perform data aggregation on the data detail table to obtain experimental data.

[0094] S207: In response to the experimental task being triggered, corresponding experimental data is acquired and the experimental program is executed to obtain experimental results.

[0095] It should be noted that the data processing principle in the embodiment of the present invention is the same as Figure 1 The corresponding data processing principles in the illustrated embodiments are the same and will not be described in detail here.

[0096] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a data processing device 300, such as Figure 3 As shown, the device 300 includes: a receiving unit 301, used to receive an experimental instruction, obtain the experimental requirements in the experimental instruction, disassemble the experimental requirements, and determine a parameter value set corresponding to each preset experimental parameter; a generating unit 302, used to match the corresponding business data table and the experimental diversion table based on the parameter value set, and associate the business data table with the experimental diversion table to generate a data detail table; an aggregation unit 303, used to extract an aggregation dimension from the parameter value set to call a preset aggregation program, perform data aggregation on the data detail table, and obtain experimental data; a processing unit 304, used to obtain corresponding experimental data in response to the experimental task being triggered and execute the experimental program in combination with the parameter value set to obtain experimental results.

[0097] It should be understood that the manner of implementing the embodiments of the present invention is different from the Figure 1 The method of the illustrated embodiment is the same and will not be repeated here.

[0098] In one embodiment, the aggregation unit 303 is specifically used to:

[0099] An aggregation target is determined based on the parameter value of the experimental indicator in the experimental parameters, and each data field associated with the aggregation target is obtained from the data detail table to extract an aggregation dimension from each of the data fields.

[0100] In yet another embodiment, the aggregation unit 303 is specifically configured to:

[0101] The diversion dimension corresponding to the experimental diversion table is obtained to update the aggregation dimension based on the diversion dimension.

[0102] In yet another embodiment, the aggregation unit 303 is specifically configured to:

[0103] Determine the correlation between each of the aggregation dimensions and the experimental indicators in the experimental parameters to sort the aggregation dimensions, and call the preset aggregation programs in sequence according to the arrangement order to perform data aggregation on the data detail table.

[0104] In yet another embodiment, the aggregation unit 303 is specifically configured to:

[0105] Query historical aggregation records based on the aggregation dimension and the data detail table;

[0106] In response to the query result being a successful query, the corresponding historical record result is determined as the experimental data.

[0107] In yet another embodiment, the receiving unit 301 is specifically configured to:

[0108] Convert the experimental requirements into text information, decompose the text information based on preset experimental parameters, determine parameter values ​​corresponding to the experimental parameters, and obtain a parameter value set;

[0109] Based on each of the parameter values, the composite parameter value of the corresponding experimental parameter is queried to update the parameter value set with the composite parameter.

[0110] It should be understood that the manner of implementing the embodiments of the present invention is different from the Figure 1 , 2 The method of the illustrated embodiment is the same and will not be repeated here.

[0111] In an embodiment of the present invention, after receiving an experimental instruction, the experimental requirements in the experimental instruction are obtained, the experimental requirements are disassembled, and the parameter value set corresponding to each preset experimental parameter is determined; then the corresponding business data table and the experimental diversion table can be matched based on the parameter value set to associate the business data table and the experimental diversion table, and the aggregation dimension can be extracted from the parameter value set to call the preset aggregation program to perform data aggregation on the data detail table to obtain the experimental data; so after the experimental task is triggered, the corresponding experimental data can be obtained and the experimental program can be executed to obtain the experimental results. In this way, in an embodiment of the present invention, the detailed data required for the corresponding experiment can be determined according to the experimental requirements and these data can be aggregated in advance, so that the experimental results can be obtained directly through the experimental data during the experiment, and the experimental data calculation link can be determined and the experimental program can be executed according to the experimental requirements, so that the experimental results can meet the experimental requirements without using the experience of the developers, thereby improving the accuracy and efficiency of the experiment.

[0112] According to an embodiment of the present invention, the embodiment of the present invention further provides an electronic device and a readable storage medium.

[0113] The electronic device of an embodiment of the present invention comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the data processing method provided by the embodiment of the present invention.

[0114] Figure 4 An exemplary system architecture 400 is shown to which the data processing method or data processing apparatus according to the embodiment of the present invention can be applied.

[0115] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, 403, network 404 and server 405. Network 404 is used to provide a medium for communication links between terminal devices 401, 402, 403 and server 405. Network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0116] The user can use the terminal devices 401, 402, 403 to interact with the server 405 through the network 404 to receive or send messages, etc. Various client applications can be installed on the terminal devices 401, 402, 403.

[0117] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0118] The server 405 may be a server that provides various services. The server may analyze and process the received data such as the product information query request, and feed back the processing result (such as product information - just an example) to the terminal device.

[0119] It should be noted that the data processing method provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the data processing device is generally set in the server 405 .

[0120] It should be understood that Figure 4 The number of terminal devices, networks and servers in the example is only for illustration purposes. Any number of terminal devices, networks and servers may be provided as required by the experiment.

[0121] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 suitable for implementing an embodiment of the present invention. Figure 5 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0122] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0123] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0124] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0125] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0126] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a unit, a program segment, or a part of a code, and the above-mentioned unit, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0127] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a receiving unit, a generating unit, an aggregation unit and a processing unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the receiving unit may also be described as a "unit of the function of the receiving unit".

[0128] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device executes the data processing method provided by the present invention.

[0129] As another aspect, the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the data processing method provided by an embodiment of the present invention.

[0130] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for data processing, characterized in that: include: Receive an experimental instruction, obtain experimental requirements in the experimental instruction, analyze the experimental requirements, and determine a parameter value set corresponding to each preset experimental parameter; Matching the corresponding business data table and the experimental diversion table based on the parameter value set, and associating the business data table with the experimental diversion table to generate a data detail table; Extracting an aggregation dimension from the parameter value set to call a preset aggregation program, perform data aggregation on the data detail table, and obtain experimental data; In response to the experimental task being triggered, corresponding experimental data is acquired and the experimental program is executed in combination with the parameter value set to obtain the experimental result.

2. The method according to claim 1, characterized in that Extracting an aggregation dimension from the parameter value set includes: An aggregation target is determined based on the parameter value of the experimental indicator in the experimental parameters, and each data field associated with the aggregation target is obtained from the data detail table to extract an aggregation dimension from each of the data fields.

3. The method according to claim 1 or 2, characterized in that: Before calling the preset aggregation program, it also includes: The diversion dimension corresponding to the experimental diversion table is obtained to update the aggregation dimension based on the diversion dimension.

4. The method according to claim 1, characterized in that: Calling a preset aggregation program to perform data aggregation on the data detail table includes: Determine the correlation between each of the aggregation dimensions and the experimental indicators in the experimental parameters to sort the aggregation dimensions, and call the preset aggregation programs in sequence according to the arrangement order to perform data aggregation on the data detail table.

5. The method according to claim 1, characterized in that The experimental requirements are analyzed to determine the parameter value set of each preset experimental parameter, including: Convert the experimental requirements into text information, decompose the text information based on preset experimental parameters, determine parameter values ​​corresponding to the experimental parameters, and obtain a parameter value set; Based on each of the parameter values, the composite parameter value of the corresponding experimental parameter is queried to update the parameter value set with the composite parameter.

6. The method according to claim 1, characterized in that Calling a preset aggregation program to perform data aggregation on the data detail table to obtain experimental data, including: Query historical aggregation records based on the aggregation dimension and the data detail table; In response to the query result being a successful query, the corresponding historical record result is determined as the experimental data.

7. A data processing device, characterized in that: include: A receiving unit, used to receive an experimental instruction, obtain an experimental requirement in the experimental instruction, analyze the experimental requirement, and determine a parameter value set corresponding to each preset experimental parameter; A generating unit, configured to match a corresponding business data table and an experimental flow distribution table based on the parameter value set, and associate the business data table with the experimental flow distribution table to generate a data detail table; An aggregation unit, used for extracting an aggregation dimension from the parameter value set to call a preset aggregation program, perform data aggregation on the data detail table, and obtain experimental data; The processing unit is used for acquiring corresponding experimental data in response to the experimental task being triggered and executing the experimental program in combination with the parameter value set to obtain the experimental result.

8. The device according to claim 7, characterized in that The polymeric unit is specifically used for: An aggregation target is determined based on the parameter value of the experimental indicator in the experimental parameters, and each data field associated with the aggregation target is obtained from the data detail table to extract an aggregation dimension from each of the data fields.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images