An online batch data testing method and system based on limited resources

Abstract

The present invention discloses an online batch data testing method and system based on limited resources. The method includes: obtaining first test data; uploading the first test data to the first test database according to a first upload instruction; generating a first resource approval form according to all task resources of a first new task; obtaining a first approval result according to the first resource approval form; if the first approval result is approval passed, obtaining a first call instruction; calling first associated test data from the first test database according to the first call instruction; and sending the first test result file to a first customer according to the first test result file generated from the first associated test data. It solves the technical problems in the prior art that there is no online standardized and perfect process for data testing before API-form interface calls, resulting in a large amount of repetitive workload and imperfect API management.

Description

Technical Field

[0001] The present invention relates to the field of big data, and particularly to an online batch data testing method and system based on limited resources. Background Art

[0002] With the continuous development of Internet and big data companies, the bank credit field has also developed accordingly, and fintech has emerged. Generally speaking, companies that master data will provide risk control services in the form of APIs to financial institutions such as banks and insurance companies by leveraging their own data advantages. Before choosing to use the risk control services of a technology company, financial institutions such as banks and insurance companies will first use some data, call the API interface, and check the quality of the returned data. Only when they are satisfied will they use it online in real time.

[0003] However, in the process of implementing the technical solution of the present invention in the embodiments of the present application, the inventors of the present application found that the above technology has at least the following technical problems:

[0004] In the prior art, there is no established online standardized and perfect process for data testing before API interface calls, which leads to a large amount of repetitive work and imperfect API management. Summary of the Invention

[0005] Aiming at the defects in the prior art, the purpose of the embodiments of the present application is to provide an online batch data testing method and system based on limited resources, which solves the technical problems in the prior art that there is no established online standardized and perfect process for data testing before API interface calls, resulting in a large amount of repetitive work and imperfect API management. It achieves the technical effect of standardizing and streamlining the data testing process, using relatively few hardware resources such as memory based on the limited API interface call frequency, completing the test data task with high quality, and improving the API management quality.

[0006] On the one hand, an embodiment of the present application provides an online batch data testing method based on limited resources. The method includes: obtaining first test data, where the first test data is all test data before API calls; uploading the first test data to the first test database according to a first upload instruction; obtaining a first new task according to a first creation instruction; generating a first resource approval form according to all task resources of the first new task; obtaining a first approval result according to the first resource approval form, where the first approval result includes a first result and a second result, the first result is approval passed, and the second result is approval not passed; if the first approval result is approval passed, obtaining a first call instruction; calling first associated test data from the first test database according to the first call instruction, where the first associated test data is the test data for API calls in the first new task; generating a first test result file according to the first associated test data, and sending the first test result file to a first customer.

[0007] On the other hand, the present application further provides an online batch data testing system based on limited resources. The system includes: a first obtaining unit: the first obtaining unit is used to obtain first test data, where the first test data is all test data before API calls; a first uploading unit: the first uploading unit is used to upload the first test data to the first test database according to a first upload instruction; a second obtaining unit: the second obtaining unit is used to obtain a first new task according to a first creation instruction; a first generating unit: the first generating unit is used to generate a first resource approval form according to all task resources of the first new task; a third obtaining unit: the third obtaining unit is used to obtain a first approval result according to the first resource approval form, where the first approval result includes a first result and a second result, the first result is approval passed, and the second result is approval not passed; a fourth obtaining unit: the fourth obtaining unit is used to obtain a first call instruction if the first approval result is approval passed; a first calling unit: the first calling unit is used to call first associated test data from the first test database according to the first call instruction, where the first associated test data is the test data for API calls in the first new task; a first sending unit: the first sending unit is used to generate a first test result file according to the first associated test data, and send the first test result file to a first customer.

[0008] In a third aspect, an embodiment of the present application provides an online batch data testing system based on limited resources, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method according to any item in the first aspect are implemented.

[0009] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0010] By obtaining all the test data before calling the API interface, the first test data is obtained and uploaded to the first test database for storage. Further, according to the first creation instruction, the first new task is obtained, and an approval form is automatically generated for all the task resources of the first new task to obtain the first approval result. If the first approval result is approval passed, the first call instruction is obtained, and according to the first call instruction, the API test data associated with the first new task is called from the first test database, and then the corresponding test file is generated based on the associated API test data and sent to the customer. It achieves the technical effect of standardizing and streamlining the process of data testing, using relatively few hardware resources such as memory based on the limited API interface call frequency, completing the test data task with high quality, and improving the API management quality.

[0011] The above description is only an overview of the technical solutions of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically described below. Description of the Drawings

[0012] By reading the detailed description of the non-restrictive embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0013] Figure 1 It is a schematic flow chart of an online batch data testing method based on limited resources in an embodiment of the present application;

[0014] Figure 2 It is a schematic flow chart of the test data management of an online batch data testing method based on limited resources in an embodiment of the present application;

[0015] Figure 3 It is a schematic flow chart of the priority judgment of an online batch data testing method based on limited resources in an embodiment of the present application;

[0016] Figure 4 It is a schematic flow chart of the priority adjustment of an online batch data testing method based on limited resources in an embodiment of the present application;

[0017] Figure 5 It is a schematic flow chart of the test data sampling of an online batch data testing method based on limited resources in an embodiment of the present application;

[0018] Figure 6 This is a schematic structural diagram of an online batch data testing system based on limited resources according to an embodiment of the present application;

[0019] Figure 7 This is a schematic structural diagram of an exemplary electronic device according to an embodiment of the present application. Detailed implementation manners

[0020] By providing an online batch data testing method and system based on limited resources, the embodiment of the present application solves the technical problems in the prior art that there is no online standardized and perfect process for data testing before API-form interface calls, resulting in a large amount of repetitive work and imperfect API management. The technical effect of completing the test data task with high quality by standardizing and streamlining the data testing process and using relatively few hardware resources such as memory based on the limited API interface call frequency, and improving the API management quality is achieved.

[0021] Next, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0022] Application overview

[0023] With the continuous development of Internet and big data companies, the bank credit field has also developed accordingly, and fintech has emerged. Generally speaking, companies that master data will provide risk control services in the form of APIs to financial institutions such as banks and insurance companies by combining their own data advantages. Before choosing to use the risk control services of a technology company, financial institutions such as banks and insurance companies will first use some data, call the API interface, and check the quality of the returned data. Only when they are satisfied will they use it online in real time. However, in the current prior art, there is no online standardized and perfect process for data testing before API-form interface calls, resulting in a large amount of repetitive work and imperfect API management.

[0024] For the above technical problems, the general idea of the technical solution provided by this application is as follows: An online batch data testing method based on limited resources is provided in an embodiment of this application. The method includes: obtaining first test data, where the first test data is all test data before API call; uploading the first test data to the first test database according to a first upload instruction; obtaining a first new task according to a first creation instruction; generating a first resource approval form according to all task resources of the first new task; obtaining a first approval result according to the first resource approval form, where the first approval result includes a first result and a second result, the first result is approval passed, and the second result is approval not passed; if the first approval result is approval passed, obtaining a first call instruction; calling first associated test data from the first test database according to the first call instruction, where the first associated test data is test data for API call in the first new task; generating a first test result file according to the first associated test data, and sending the first test result file to a first customer.

[0025] To better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0026] Embodiment 1

[0027] As Figure 1 shown, an embodiment of this application provides an online batch data testing method based on limited resources. The method includes:

[0028] Step S100: Obtain first test data, where the first test data is all test data before API call;

[0029] Step S200: Upload the first test data to the first test database according to a first upload instruction;

[0030] Furthermore, for the task support management of the first test database in step S200 of the embodiment of this application, start, pause, resume, priority, speed control, and task progress display are supported, and split execution of tasks and human intervention management are supported.

[0031] Specifically, companies in the Internet and big data sectors, as well as specific government departments and agencies, will have some proprietary data or data purchased from third parties as their business grows and they seek to make a profit. After encrypting and desensitizing the data, they will provide some data query services to the outside world, such as real-name authentication services for ID cards and names, and SMS sending services. The service provider provides an API interface to the public customers, opens its services, and charges according to the number of calls (times), etc. This application provides an online batch data testing method based on limited resources. Further, all the test data of the API interface calls can be obtained first, that is, the first test data, and the first test data supports files in multiple formats such as Excel, CSV, and TXT during the upload process, supports large files of 2G, 100 million, etc., and supports multiple forms of upload by customers through the system online, FTP upload, and internal staff upload. Thus, all the uploaded data is stored in the first test database for management, and the corresponding management supports multiple management modes. Furthermore, the entire process of data testing can be standardized and streamlined, and the fully online system saves communication costs and operation costs.

[0032] Step S300: Obtain a first new task according to the first creation instruction.

[0033] Step S400: Generate a first resource approval form according to all the task resources of the first new task.

[0034] Furthermore, in step S300 of the embodiment of this application, the speed control of all new tasks is achieved by adjusting the frequency of the underlying API, the number of supported threads, and the time period, dynamically adjusting the overall execution of the system tasks.

[0035] Specifically, the first new task is created by integrating resources such as selected test data, account passwords, products, and quantities. Further, before executing the first new task, the user needs to prepare the permissions for the resources, including account passwords, products, and quantities used for API interface calls and other resources related to task execution, so as to generate a corresponding approval work order for permission approval. Further, the security before resource execution is increased. An API (Application Programming Interface) is a set of predefined interfaces (such as functions, HTTP interfaces), or an agreement on the connection of different components of a software system. It is used to provide a set of routines for an application program and developers to access based on a certain software or hardware, without the need to access the source code or understand the details of the internal working mechanism. Generally speaking, creating a task requires applying for an account to be opened, and applying for resource permissions such as products and quantities required for API interface calls. Therefore, the application for permissions is achieved by generating a corresponding resource approval form.

[0036] Furthermore, the speed control of all newly created tasks dynamically adjusts the overall execution of system tasks by adjusting the frequency of the underlying API, the number of supported threads, and the time period. And in the process of constructing a new task, test data files, products, quantities, accounts and other resources are selected to initiate the task, and then wait for execution. When there are a large number of newly created tasks, speed control needs to be performed on the process of creating and executing tasks. For example, by analyzing the frequency, number of threads, and time period of the underlying API, the execution speed of the first newly created file is dynamically adjusted.

[0037] Step S500: Obtain a first approval result according to the first resource approval form, where the first approval result includes a first result and a second result, the first result is approval passed, and the second result is approval not passed;

[0038] Step S600: If the first approval result is approval passed, obtain a first call instruction;

[0039] Specifically, the first resource approval form is an accurate process of user resources. When the first resource approval form is approved, that is, the first result, a first execution instruction is obtained to implement the execution of the first newly created task. If the first resource approval form is not approved, that is, the second result, it is correspondingly detected whether the first resource approval form contains non-authorization application data. Further, when the first resource approval form is approved, further by analyzing the commonness of tasks, after meeting permissions and conditions such as priorities, according to a certain mechanism, all test data associated with the task is read to ensure approval passed and achieve the security of task execution.

[0040] Step S700: Call first associated test data from the first test database according to the first call instruction, where the first associated test data is the test data for API calls in the first newly created task;

[0041] Step S800: Generate a first test result file according to the first associated test data and send the first test result file to a first customer.

[0042] Specifically, after the first approval form is approved, after the result data is obtained by calling the API interface respectively and the task is successfully executed, for all test data, each API call will generate a test result; at the entire task level, analysis and summary will be performed to obtain several total result files. For example, various information such as how many times it was successful and how many times it failed will be recorded. And the first associated test data is the test data based on the API calls in the first newly created task. Further, the process of executing the first newly created task can be understood as reading out the associated test data one by one, calling the API interface respectively, obtaining the test data, and thus generating the corresponding test result file.

[0043] Furthermore, for one of the tasks, the file data is read according to a certain strategy to generate a result once. The result data is also divided into a message queue for producing data and a message queue for consuming data. The consumption queue is to summarize the result files of each piece of data. For example, count the number of successful times, and check whether the data is correct, etc. After the relevant user knows the task result, a report will be generated according to their own understanding and the customer's needs. After the customer receives the report, feedback will be given; if satisfied, the entire task is considered to be completed. If it involves paid API calls, payment will be made according to the bill. It achieves the technical effects of standardizing and streamlining the process through the data testing process, using relatively less memory and other hardware resources based on the limited frequency of API interface calls, completing the test data task with high quality, and improving the API management quality.

[0044] Further, as Figure 2 shown, in the step S200 of the embodiment of the present application, uploading the first test data to the first test database according to the first upload instruction further includes:

[0045] Step S210: Obtain the first upload form and the first file format of the first test data, where the first test data supports multiple file formats and multiple upload forms;

[0046] Step S220: Generate a dual-attribute storage label according to the first upload form and the first file format;

[0047] Step S230: Perform a standardization inspection on the first test database according to the dual-attribute storage label to obtain first inspection information;

[0048] Step S240: Manage the first test data according to the first inspection information.

[0049] Specifically, the first upload form of the first test data is the format for uploading the first test data, which supports various forms such as online upload by customers through the system, FTP upload, and upload by internal personnel. On the other hand, the first file format of the first test data is a file in various formats such as Excel, CSV, TXT, etc., or large files such as 2G and 100 million. By analyzing the attribute tag fields of the first test data, the corresponding dual-attribute storage tags are generated to facilitate the management of the first test data in the first test database, and the tasks of the test data support management, start, pause, resume, priority, speed control, and task progress display. It supports the split execution of tasks and human intervention management. Further, before data upload, the retrieved data in the first test database is inspected to obtain the corresponding first inspection information. For example, the file format, encryption method, and data permissions are verified. If there are problems, an error report will be generated, which records some error information. The first test database is managed according to the corresponding error information, and the complete process of data testing is standardized and streamlined. The entire system is online, saving communication costs and operation costs.

[0050] Further, as Figure 3 shown, wherein, before obtaining the first call instruction when the first approval result is approval passed, step S600 of the embodiment of the present application further includes:

[0051] Step S610: Determine whether the first newly created task is in the common task list;

[0052] Step S620: If the first newly created task is in the common task list, obtain the first task priority of the first newly created task;

[0053] Step S630: By judging the level permissions of the first task priority, if the level permissions of the first task priority meet the preset level permissions, obtain the first call instruction.

[0054] Specifically, according to the quantity of test data, the priorities of internal tasks, system hardware resources, global resources of the system's underlying API, etc., the execution order between system tasks is adjusted. The execution process of a task is that when the task is common and meets the permissions and conditions such as priorities, according to a certain mechanism, all the test data associated with the task is read, and file data is read according to a certain strategy. For example, if the API is called conditionally 10 times, 10 lines of data are read. First, the 10 lines of data are written into the message queue of production data; then a message queue for consuming data is maintained. The consumption process is to read one piece of data, call the API, and generate one result. Under the combined action of several message queues, multiple final result files are finally obtained. Further, when the level permission of the first newly created task meets the preset level permission, the first call instruction is obtained, and then the corresponding associated test data is called, achieving the process standardization and standardization of the execution of the newly created task based on priority judgment.

[0055] Further, as Figure 4 shown, when the first newly created task is in the common task list, after obtaining the first task priority of the first newly created task, step S620 of the embodiment of the present application further includes:

[0056] Step S621: Obtain the quantity information of the first associated test data in the first newly created task;

[0057] Step S622: Obtain the internal priority of the first newly created task, where the internal priority is the priority of the internal task in the task list;

[0058] Step S623: Obtain the system resource information of the first newly created task;

[0059] Step S624: Input the quantity information, the internal priority, and the system resource information into the priority order adjustment model, and obtain the first adjustment information according to the priority order adjustment model, where the priority order adjustment model is trained with multiple groups of data until convergence;

[0060] Step S625: Adjust the order of the first task priority according to the first adjustment information.

[0061] Specifically, the priority order adjustment model is a machine learning model, and a neural network (Neural Networks, NN) is a complex neural network system formed by a large number of simple processing units (called neurons) widely interconnected with each other. It reflects many basic characteristics of the human brain function and is a highly complex non-linear dynamic learning system. The neural network model is described based on the mathematical model of neurons. An artificial neural network (Artificial Neural Networks, ANN) is a description of the first-order characteristics of the human brain system. Simply put, it is a mathematical model. Through the training of a large amount of training data, the training data is obtained by training multiple groups of training data until convergence. The multiple groups of data include the quantity information, the internal priority, and the system resource information.

[0062] Furthermore, the process of the training is essentially a process of supervised learning. The quantity information, the internal priority, and the system resource information are input into the priority order adjustment model. The neural network model continuously self-corrects and adjusts until the first training result obtained is consistent with the identification information, ending the supervised learning of this group of data and proceeding to the supervised learning of the next group of data; when the output information of the neural network model reaches a predetermined accuracy rate / reaches a convergence state, the supervised learning process ends. Through the supervised learning of the neural network model, the neural network model can process the input information more accurately, and thus the lowest charging amount in the input information output is more reasonable and accurate, thereby achieving intelligent priority adjustment and improving the technical effect of API management quality.

[0063] Further, as Figure 5 shown, step S300 of the embodiment of the present application further includes:

[0064] Step S310: Obtain all task data volumes of the first newly created task;

[0065] Step S320: Through performing data quantization analysis on all the task data volumes, obtain a first data quantization index;

[0066] Step S330: If the first data quantization index is not within the preset data quantization index, obtain a first sampling instruction;

[0067] Step S340: Sample from all the task data volumes according to the first sampling instruction to obtain first sampling data;

[0068] Step S350: Use the first sampling data as the priority execution data of the first newly created task.

[0069] Specifically, after the user uploads the test file, the file is not read into the memory all at once, nor are all the data used simultaneously. Instead, the API interface is called to obtain the result data in a way that reads only as much as needed. For example, if there are resources available to execute API calls for 10 pieces of data, then 10 pieces are read. The index positions of the already-read data (from 1 to 10) are maintained in the memory and the database. When the task is in progress and after it is completed, the status corresponding to the index (from 1 to 10, read, completed) is maintained. Before executing a task with a large amount of data, it is supported to extract and execute a part of the data first (sequentially, randomly, proportionally, etc.) to save costs. Although the data volume is very large, through the file indexing technology, by reading only as much as needed, only a small amount of memory is occupied to complete the execution of the task. Before testing all the data, it is supported to extract a part of the data and execute it first according to a certain method. After preliminary verification, then execute the test task for all the data. The scenarios involved in sampling include extracting 100 pieces from tens of millions of test data; data verification, data execution permissions, pre-sampling verification to prevent errors during execution; data testing, the user's own data is free, but if third-party data or interfaces are needed, a fee is required.

[0070] For further sampling execution, to avoid data errors and poor effects, when executing all the data, it will cause waste. And the sampling algorithms include sequential extraction, random extraction, specified extraction, and proportional extraction. Among them, for the sequential extraction, for example, the first 100 or the last 100 or 100 starting from a certain position out of 10 million pieces are extracted; for the random extraction, after specifying the number of pieces, 100 pieces of data are randomly extracted from all the data; for the specified extraction, the user specifies the positions of the data, for example: for 10 pieces of data, the data on the 1st, 3rd, and 5th rows; for the proportional extraction, 10 pieces need to be extracted from 100 pieces of data, and 1 piece is extracted every 10 rows. This avoids large-scale errors when executing all the data and saves resources. It achieves the technical effect of using relatively few hardware resources such as memory to complete the test data task with high quality under the condition that the API interface call frequency is limited.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] 1. By obtaining all the test data before calling the API interface, the first test data is obtained and uploaded to the first test database for storage. Further, according to the first creation instruction, the first new task is obtained, and an approval form is automatically generated for all the task resources of the first new task to obtain the first approval result. If the first approval result is approval passed, the first call instruction is obtained, and according to the first call instruction, the API test data associated with the first new task is called from the first test database, so as to generate the corresponding test file according to the associated API test data and send it to the customer at the underlying API interface. Under limited conditions such as the number of threads and call frequency, the existing business resource conditions are fully utilized to maximize the efficiency of accelerating the data test task.

[0073] 2. Before executing a task with a large amount of data, it is supported to extract part of the data for execution (sequentially, randomly, proportionally), avoiding large-area errors when all the data is executed, saving resources and costs.

[0074] 3. The complete process of data testing is processed in a standardized and process-oriented manner, and the entire system is online, saving communication costs and operation costs.

[0075] Embodiment 2

[0076] Based on the same inventive concept as a method for online batch data testing based on limited resources in the foregoing embodiments, the present invention also provides an online batch data testing system based on limited resources, as Figure 6 shown, the system includes:

[0077] The first obtaining unit 11: The first obtaining unit 11 is used to obtain the first test data, where the first test data is all the test data before API call;

[0078] The first uploading unit 12: The first uploading unit 12 is used to upload the first test data to the first test database according to the first upload instruction;

[0079] The second obtaining unit 13: The second obtaining unit 13 is used to obtain the first new task according to the first creation instruction;

[0080] The first generating unit 14: The first generating unit 14 is used to generate the first resource approval form according to all the task resources of the first new task;

[0081] The third obtaining unit 15: The third obtaining unit 15 is used to obtain the first approval result according to the first resource approval form, where the first approval result includes the first result and the second result, the first result is approval passed, and the second result is approval not passed;

[0082] Fourth acquisition unit 16: The fourth acquisition unit 16 is configured to obtain a first call instruction if the first approval result is approval passed.

[0083] First call unit 17: The first call unit 17 is configured to call first associated test data from the first test database according to the first call instruction, where the first associated test data is test data for API calls in the first new task.

[0084] First sending unit 18: The first sending unit 18 is configured to generate a first test result file according to the first associated test data and send the first test result file to a first customer.

[0085] Further, the system further includes:

[0086] Fifth acquisition unit: The fifth acquisition unit is configured to obtain a first upload form and a first file format of the first test data, where the first test data supports multiple file formats and multiple upload forms.

[0087] Second generation unit: The second generation unit is configured to generate a dual-attribute storage label according to the first upload form and the first file format.

[0088] Sixth acquisition unit: The sixth acquisition unit is configured to perform a standardization check on the first test database according to the dual-attribute storage label to obtain first check information.

[0089] First management unit: The first management unit is configured to manage the first test data according to the first check information.

[0090] Further, the system further includes:

[0091] First judgment unit: The first judgment unit is configured to judge whether the first new task is in a common task list.

[0092] Seventh acquisition unit: The seventh acquisition unit is configured to obtain a first task priority of the first new task if the first new task is in the common task list.

[0093] Eighth acquisition unit: The eighth acquisition unit is configured to judge the level permission of the first task priority, and if the level permission of the first task priority meets a preset level permission, obtain the first call instruction.

[0094] Further, the system further includes:

[0095] The ninth acquisition unit: The ninth acquisition unit is used to acquire the quantity information of the first associated measurement data in the first newly created task;

[0096] The tenth acquisition unit: The tenth acquisition unit is used to acquire the internal priority of the first newly created task, where the internal priority is the priority of the internal tasks in the task list;

[0097] The eleventh acquisition unit: The eleventh acquisition unit is used to acquire the system resource information of the first newly created task;

[0098] The first input unit: The first input unit is used to input the quantity information, the internal priority, and the system resource information into the priority order adjustment model, and obtain first adjustment information according to the priority order adjustment model, where the priority order adjustment model is trained with multiple groups of data until convergence;

[0099] The first adjustment unit: The first adjustment unit is used to adjust the order of the first task priority according to the first adjustment information.

[0100] Furthermore, the system further includes:

[0101] The twelfth acquisition unit: The twelfth acquisition unit is used to acquire all task data volumes of the first newly created task;

[0102] The thirteenth acquisition unit: The thirteenth acquisition unit is used to obtain a first data quantization index by performing execution data quantization analysis on all task data volumes;

[0103] The first estimation unit: The first estimation unit is used to obtain a first sampling instruction if the first data quantization index is not within the preset data quantization index;

[0104] The fourteenth acquisition unit: The fourteenth acquisition unit is used to sample from all task data volumes according to the first sampling instruction to obtain first sampling data;

[0105] The first operation unit, the first operation unit is used to use the first sampling data as the priority execution data of the first newly created task.

[0106] The foregoing Figure 1 All the various change methods and specific examples of the online batch data testing method based on limited resources in Embodiment 1 are equally applicable to the online batch data testing system based on limited resources in this embodiment. Through the foregoing detailed description of the online batch data testing method based on limited resources, those skilled in the art can clearly know the implementation method of the online batch data testing system based on limited resources in this embodiment. Therefore, for the sake of simplicity of the specification, it will not be elaborated here.

[0107] Embodiment 3

[0108] The following will describe the electronic device according to the embodiments of the present application with reference to Figure 7 to describe the electronic device according to the embodiments of the present application.

[0109] Figure 7 The structural schematic diagram of the electronic device according to the embodiments of the present application is illustrated.

[0110] Based on the inventive concept of a method for testing online batch data based on limited resources in the foregoing embodiments, the present invention further provides an online batch data testing system based on limited resources, on which a computer program is stored, and when the program is executed by a processor, it implements the steps of any of the methods of the online batch data testing system based on limited resources described above.

[0111] Among them, in Figure 7 , the bus architecture (represented by bus 300), bus 300 may include any number of interconnected buses and bridges, and bus 300 links together various circuits including one or more processors represented by processor 302 and a memory represented by memory 304. Bus 300 may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, etc., which are well known in the art, and thus will not be further described herein. Bus interface 305 provides an interface between bus 300 and receiver 301 and transmitter 303. Receiver 301 and transmitter 303 may be the same element, i.e., a transceiver, providing a unit for communicating with various other systems on a transmission medium. Processor 302 is responsible for managing bus 300 and general processing, and memory 304 may be used to store data used by processor 302 when performing operations.

[0112] An embodiment of the present application provides an online batch data testing method based on limited resources. The method includes: obtaining first test data, where the first test data is all test data before API call; uploading the first test data to the first test database according to a first upload instruction; obtaining a first new task according to a first creation instruction; generating a first resource approval form according to all task resources of the first new task; obtaining a first approval result according to the first resource approval form, where the first approval result includes a first result and a second result, the first result is approval passed, and the second result is approval not passed; if the first approval result is approval passed, obtaining a first call instruction; calling first associated test data from the first test database according to the first call instruction, where the first associated test data is test data for API call in the first new task; generating a first test result file according to the first associated test data, and sending the first test result file to a first customer. It solves the technical problem in the prior art that there is no established online standard and perfect process for data testing before API interface call, resulting in a large amount of repetitive work and imperfect API management. It achieves the technical effect of standardizing and streamlining the data testing process, using relatively few hardware resources such as memory based on the limited API interface call frequency, completing the test data task with high quality, and improving the API management quality.

[0113] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a system for realizing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system that implements the function specified in one or more of the processes and / or blocks Figure 1 in the process or processes and / or blocks Figure 1 specified in the block or blocks.

[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the function specified in one or more of the processes and / or blocks Figure 1 in the process or processes and / or blocks Figure 1 specified in the block or blocks.

[0117] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made by those skilled in the art once they learn of the basic inventive concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the present invention.

[0118] It is apparent that those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. An online batch data testing method based on limited resources, wherein, The method includes: Obtain first test data, where the first test data is all test data before API call; Upload the first test data to a first test database according to a first upload instruction; Obtain a first new task according to a first creation instruction; Generate a first resource approval form according to all task resources of the first new task; Obtain a first approval result according to the first resource approval form, where the first approval result includes a first result and a second result, the first result is approval passed, and the second result is approval not passed; If the first approval result is approval passed, obtain a first call instruction; Call first associated test data from the first test database according to the first call instruction, where the first associated test data is test data for API call in the first new task; Generate a first test result file according to the first associated test data and send the first test result file to a first customer; The method further includes: Obtain all task data volumes of the first new task; Obtain a first data quantization index by performing data quantization analysis on all the task data volumes; If the first data quantization index is not within a preset data quantization index, obtain a first sampling instruction; Perform sampling from all the task data volumes according to the first sampling instruction to obtain first sampling data; Use the first sampling data as the priority execution data of the first new task.

2. The method according to claim 1, wherein, When uploading the first test data to the first test database according to the first upload instruction, the method further includes: Obtain a first upload form and a first file format of the first test data, where the first test data supports multiple file formats and multiple upload forms; Generate a dual-attribute storage label according to the first upload form and the first file format; Perform a standardized inspection on the first test database according to the dual-attribute storage label to obtain first inspection information; Manage the first test data according to the first inspection information.

3. The method according to claim 2, wherein, The tasks of the first test database support management, start, pause, resume, priority, speed control, task progress display, support for split execution of tasks, and human intervention management.

4. The method according to claim 1, wherein Before obtaining the first call instruction if the first approval result is approval passed, the method further includes: Judge whether the first new task is in a common task list; If the first new task is in the common task list, obtain a first task priority of the first new task; Judge the level permissions of the first task priority. If the level permissions of the first task priority meet the preset level permissions, obtain the first call instruction.

5. The method according to claim 4, wherein After obtaining the first task priority of the first new task if the first new task is in the common task list, the method further includes: Obtain quantity information of the first associated test data in the first new task; Obtain an internal priority of the first new task, where the internal priority is the priority of tasks inside the task list; Obtain the system resource information of the first newly created task; Input the quantity information, the internal priority, and the system resource information into the priority adjustment model, and obtain first adjustment information according to the priority adjustment model, wherein the priority adjustment model is trained with multiple groups of data until convergence; Adjust the order of the first task priority according to the first adjustment information.

6. The method according to claim 1, wherein The speed control of all newly created tasks dynamically adjusts the overall execution of system tasks by adjusting the frequency of the underlying API, the number of supported threads, and the time period.

7. An online batch data testing system based on limited resources, wherein, The system includes: The first obtaining unit: The first obtaining unit is used to obtain first test data, wherein the first test data is all test data before API call; The first uploading unit: The first uploading unit is used to upload the first test data to the first test database according to the first upload instruction; The second obtaining unit: The second obtaining unit is used to obtain a first newly created task according to the first creation instruction; The first generating unit: The first generating unit is used to generate a first resource approval form according to all task resources of the first newly created task; The third obtaining unit: The third obtaining unit is used to obtain a first approval result according to the first resource approval form, wherein the first approval result includes a first result and a second result, the first result is approval passed, and the second result is approval not passed; The fourth obtaining unit: The fourth obtaining unit is used to obtain a first call instruction if the first approval result is approval passed; The first calling unit: The first calling unit is used to call first associated test data from the first test database according to the first call instruction, wherein the first associated test data is the test data of API call in the first newly created task; The first sending unit: The first sending unit is used to generate a first test result file according to the first associated test data and send the first test result file to the first customer; The system further includes: The twelfth obtaining unit: The twelfth obtaining unit is used to obtain all task data volumes of the first newly created task; The thirteenth obtaining unit: The thirteenth obtaining unit is used to obtain a first data quantization index by performing execution data quantization analysis on all task data volumes; The first estimation unit: The first estimation unit is used to obtain a first sampling instruction if the first data quantization index is not within the preset data quantization index; The fourteenth obtaining unit: The fourteenth obtaining unit is used to sample from all task data volumes according to the first sampling instruction to obtain first sampling data; The first operation unit, and the first operation unit is used to use the first sampling data as the priority execution data of the first newly created task.

8. An online batch data testing system based on limited resources, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, When the processor executes the program, it implements the steps of the method according to any one of claims 1-6.

Images