Automatic generation method and related device for data processing flow test report

By creating task inspection objects for each node of the high-precision map data processing process and generating test reports, the problem of difficult testing in complex processes is solved, automated testing is achieved, and efficiency and accuracy are improved.

CN111930606BActive Publication Date: 2024-11-22ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910394979.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-05-13
Publication Date
2024-11-22
Estimated Expiration
2039-05-13

AI Technical Summary

Technical Problem

The high-precision map data processing process test is complex and irreversible, which leads to the quality assurance engineers facing cumbersome and long data processing processes, making it difficult to ensure the correctness of each node and the normal operational tasks generated by the final generation. Especially in the software rapid iteration cycle, automation is required. Means to support the data processing process testing of large data volumes.

Method used

Create a task inspection object for each node in the data processing process, and update its test result information after the test is completed, generate a test report for all or some nodes, and automatically count and generate tests through the task inspection object list and attribute information. Report.

Benefits of technology

It realizes automated testing of the data processing process, avoids the omission of manual inspection, improves testing efficiency, reduces testing time, and is suitable for process nodes of various complexity levels.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111930606B_ABST
    Figure CN111930606B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for automatically generating a data processing flow test report and a related device. The method comprises: creating at least one corresponding task inspection object for each flow node in the data processing flow, and after the test of each flow node in the data processing flow is completed, using the test result information of each flow node to update the corresponding task inspection object; obtaining a list of at least one task inspection object corresponding to each flow node in the data processing flow; and generating a test report of the data processing flow according to the test result information in the task inspection object. The present invention can automatically generate a test report of the data processing flow, providing an automated support means for QA, which avoids the possibility of omissions in manual inspection on the one hand, and also improves the overall efficiency of the test and reduces the test time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer testing technology, and in particular to a method for automatically generating a data processing flow test report and a related device. Background Art

[0002] Some data processing flow tests, such as HD map data flow test projects, are characterized by long processes, serious data dependence between modules, and often irreversible processes, which makes single-point testing very complicated. Moreover, the original input data of HD maps, including data for interaction between processes, cannot be constructed by humans (such as trajectory data, point cloud data, photo data, etc.), which also makes data flow testing very difficult. During the HD map data flow testing process, quality assurance engineers (QA, Quality Assurance) face a large amount of complex unknown test data and tedious and lengthy data processing processes. The testing process is relatively difficult, as it is necessary to ensure that the processing of each node in the data processing flow is correct and that the final generated operational tasks are normal. With the development of the entire project, the quality output delivered by QA for each iteration cycle is no longer guaranteed by simply testing some small data. Faced with the rapid iteration cycle of software (1-2 weeks), in addition to testing multiple targeted data in each iteration, data regression testing is also required for the fixed bugs. At the same time, the technical implementation of data flow is irreversible, which means that in most cases, problems encountered during testing require re-running the process after bugs are fixed.

[0003] As a qualified QA, you cannot treat the entire data processing process as a black box for testing. During the testing process, the flow, execution time, and functional correctness of each step of the process need to be covered by the test. However, for a medium-sized data collection data processing process alone, which contains a large number of process nodes, the process tasks divided by the data processing can reach hundreds. It is unrealistic to manually check and verify all these hundreds of process tasks. However, if a sampling verification method is used, there may be omissions, resulting in the inability to guarantee quality in the end. Therefore, there is an urgent need for automated means to support the testing of data processing processes with large amounts of data. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide a method and related device for automatically generating a data processing flow test report that overcomes the above problems or at least partially solves the above problems.

[0005] In a first aspect, an embodiment of the present invention provides a method for automatically generating a data processing flow test report, comprising:

[0006] Creating at least one corresponding task inspection object for each process node in the data processing process, and after the test of each process node in the data processing process is completed, using the test result information of each process node to update the corresponding task inspection object;

[0007] Obtain a list of at least one task inspection object corresponding to each process node in the data processing process;

[0008] A test report of the data processing flow is generated based on the test result information recorded in the task inspection object.

[0009] In one embodiment, generating a test report of the data processing flow according to the test result information in the task inspection object includes:

[0010] According to the test result information recorded in the task inspection object, the full test results of the entire data processing process and / or the test results of each process node are counted;

[0011] Generate a corresponding test report based on the statistically obtained full test results of the entire data processing flow and / or the test results of each process node.

[0012] In one embodiment, the task check object includes at least one of the following: database result object attributes, common attributes, and checkpoint attributes;

[0013] After the test of each process node of the data processing process is completed, the task inspection object is updated using the test result information of each process node, including:

[0014] After the process node test of the data processing process is completed, the database result object attributes in the created task check object are updated and assigned according to the information recorded in the database for each process node;

[0015] According to the running test data stored in the data processing system running the data processing flow, the common attributes and checkpoint attributes in the created task check object are updated and assigned values.

[0016] In one embodiment, at least one corresponding task inspection object is created for each process node in the data processing process, specifically including:

[0017] Determine the type of the task inspection object according to the type of the corresponding process node;

[0018] Determine the number of data processing tasks included in the process node according to the number of business data files and business data folders corresponding to each process node in the data processing system, or according to the number of data processing results of the previous process node of the process node, and determine the number of corresponding task inspection objects according to the determined number of data processing tasks;

[0019] According to the determined type and quantity of the task inspection object, at least one task inspection object of the corresponding type and the corresponding quantity is created.

[0020] In one embodiment, the common attribute includes one or more of the following attribute information: the overall inspection result of all test points corresponding to the task inspection object; the total number of all test points corresponding to the entire task inspection object, the number of test points corresponding to the task inspection object that have successfully run, and the number of test points corresponding to the task inspection object that have failed to run;

[0021] The checkpoint attributes include attribute information of the test point itself.

[0022] In one embodiment, the task check object is a Java object, and the test result information in the task check object is obtained from the Java object through a Java reflection mechanism.

[0023] In a second aspect, an embodiment of the present invention provides an automatic generation device for a data processing flow test report, comprising:

[0024] A task check object generation module, used to create at least one corresponding task check object for each process node in the data processing process, and after the test of each process node in the data processing process is completed, update the corresponding task check object using the test result information of each process node;

[0025] An acquisition module, used to acquire a list of at least one task inspection object corresponding to each process node in the data processing process;

[0026] A report generation module is used to generate a test report of the data processing flow according to the test result information recorded in the task inspection object.

[0027] In one embodiment, the report generation module is specifically used to count the full test results of the entire data processing process and / or the test results of each process node based on the test result information recorded in the task inspection object; and generate a corresponding test report based on the full test results of the entire data processing process and / or the test results of each process node obtained by statistics.

[0028] In a third aspect, an embodiment of the present invention provides a test server, comprising: a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the instructions, when executed by the processor, can implement the aforementioned method for automatically generating a data processing flow test report.

[0029] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implements the method for automatically generating a data processing flow test report as described above.

[0030] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0031] The automatic generation method and related device of the data processing flow test report provided by the embodiment of the present invention pre-creates at least one task inspection object for the data processing task of each process node in the data processing flow, and after the test of each process node in the data processing flow is completed, the corresponding task inspection object is updated using the test result information of each process node; when generating the test report, first obtain a list of at least one task inspection object corresponding to each process node in the data processing, and then generate the corresponding data flow test report according to the test result information in the inspection task object obtained. The present invention designs at least one corresponding task inspection object for the task of each process node, and this or these task inspection objects can realize the automatic inspection of the test results of the process node, so as to avoid testing the entire complex data processing flow as a black box, and the quality of each process node in the flow can be globally controlled through the test report. Through the test result information in the task inspection object, the test report of the data processing flow can be automatically generated, providing an automated support means for QA, on the one hand, avoiding the possibility of omissions in manual inspection, and on the other hand, improving the overall efficiency of the test and reducing the test time.

[0032] On the other hand, no matter how complex the data processing process is or how many process nodes there are, the method provided by the embodiment of the present invention can be used to automatically generate a test report, which has wide applicability and good scalability.

[0033] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0034] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0036] Figure 1 A flowchart of a method for automatically generating a data processing flow test report in an embodiment of the present invention;

[0037] Figure 2 A flowchart of the steps for generating a test report of the data processing flow in an embodiment of the present invention;

[0038] Figure 3 A flowchart of an example of a method for automatically generating a data processing flow test report in an embodiment of the present invention;

[0039] Figure 4 A flowchart of the steps of generating a task inspection object in an embodiment of the present invention;

[0040] Figure 5 is another flow chart of an example of a method for automatically generating a data processing flow test report in an embodiment of the present invention;

[0041] Figure 6 and Figure 7 This is an example of a data segmentation report in an embodiment of the present invention;

[0042] Figure 8 Schematic diagram of the structure of the automatic generation device of the data processing flow test report in the embodiment of the present invention. DETAILED DESCRIPTION

[0043] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0044] The automatic generation method of the data flow test report provided by the embodiment of the present invention is based on the white box testing method. White box testing is also called structural testing, transparent box testing, logic-driven testing or code-based testing. White box testing is a test case design method. The box refers to the software being tested, and the white box means that the box is visible, and it is clear what is inside the box and how it works. Therefore, the "white box" testing method requires a comprehensive test of the internal part of the data processing flow being tested.

[0045] For the data processing process, it includes many process nodes. Taking the data flow test of high-precision maps as an example, some nodes are responsible for processing photo data, some nodes are responsible for processing trajectory data, some nodes are responsible for processing point cloud data, etc. Each process node is implemented as a module. These process nodes have corresponding inputs and outputs and are interdependent.

[0046] Based on this, the automatic generation method and related device of the data processing flow test report provided in the embodiment of the present invention automatically obtain the test report by processing the test results of each process node in the data processing flow, so as to fully display the test status of each link in the data processing flow, so that QA can quickly determine whether the function of the tested data processing flow is correct by reading the test report.

[0047] The following describes in detail the specific implementation of the method for automatically generating a data processing flow test report and related devices provided by the embodiments of the present invention in conjunction with the accompanying drawings.

[0048] The method for automatically generating a data processing flow test report provided by the embodiment of the present invention refers to Figure 1 As shown, the following steps are included:

[0049] S11, creating at least one corresponding task inspection object for each process node in the data processing process;

[0050] S12. After the test of each process node of the data processing process is completed, the test result information of each process node is used to update the corresponding task inspection object;

[0051] S13, obtaining a list of at least one task inspection object corresponding to each process node in the data processing process;

[0052] S14. Generate a test report for the data processing flow based on the test result information recorded in the task inspection object.

[0053] The inventor of the present invention discovered that in the process of developing an automated testing program that can automatically generate test reports, how can the test reports be made more comprehensive, smarter, and more elegant? At the same time, the method must also maintain high maintainability and scalability, and be easier to develop. It can be implemented based on Java, that is, corresponding Java objects are designed for each data processing task (job) of the process node in the data processing process, which are called task check objects. In order to simplify the explanation, the task check object is referred to as the job object below.

[0054] In the above step S12, after the test of each process node of the data processing flow is completed, the test result information of each process node is used to update the corresponding task inspection object. These process nodes can be updated after the test of all process nodes in the entire data processing flow is completed, and the task inspection object can be updated in time after the test of one of the process nodes is completed, without having to wait for the test of all process nodes to be completed.

[0055] In one embodiment, in the above step S14, a test report of the data processing flow is generated according to the test result information in the task inspection object, referring to Figure 2 As shown, it can be realized in the following ways:

[0056] S21. According to the test result information recorded in the task inspection object, the full test results of the entire data processing process and / or the test results of each process node are counted;

[0057] S22. Generate a corresponding test report based on the statistically obtained full test results of the entire data processing flow and / or the test results of each flow node.

[0058] In the above step S21, based on the test result information in at least one task check object corresponding to each process node, the test results of all process nodes in the entire data processing process can be summarized, for example, the total number of test points (i.e., checkpoints) in the data processing process, the number of successful ones, and the number of failed ones. The test results of each process node in the data processing process can also be summarized separately to provide a corresponding report, which can include only one of the above two or both.

[0059] Reference Figure 3 For the example shown, the overall process is as follows:

[0060] Step 1: First load the job object list corresponding to each process node (module);

[0061] Step 2: Calculate the test results of the entire data processing flow;

[0062] Step 3: Calculate the test results of each process node (module);

[0063] Step 4: Output the test report according to the whole or process node (module);

[0064] Step 5: Determine whether to send an email notification. If yes, proceed to step 6.

[0065] Step 6: Execute the steps of email notification;

[0066] Step 7: Whether to perform backup, if yes, proceed to step 8;

[0067] Step 8: Back up the report according to the iterative version.

[0068] There is no strict execution order for the above steps 2 and 3, and they can be processed in parallel or executed one after the other, and step 2 can be executed before step 3, or vice versa.

[0069] In one embodiment, the above steps S11 and S12 can be implemented by referring to the process shown in Figure 4:

[0070] S41, creating at least one task check object for each process node; the task check object includes at least one of the following: database result object attributes, common attributes, and checkpoint attributes;

[0071] S42, after the process node test of the data processing process is completed, the created database result object attributes are updated and assigned according to the information recorded in the database for each process node;

[0072] S43. Update and assign values ​​to the created common attributes and checkpoint attributes according to the running test data stored in the data processing system running the data processing flow.

[0073] In the process of generating the at least one task inspection object mentioned above, in step S41, at least one corresponding task inspection object is first created according to the type of process node. For example, when the job object is initially created, it contains multiple object attributes, such as database result object attributes, public attributes and checkpoint attributes, as well as the attributes of the job object itself.

[0074] Different types of process nodes have corresponding job object types of different types. Different parts have separate classes to maintain them, but they also contain many common parts, which are abstracted and encapsulated into specific methods.

[0075] The created task check object has all its attributes set to default values ​​at the beginning. After the test of the corresponding process node is completed, the test results of each data processing task will be checked, and the initial default attribute values ​​at the time of creation will be updated accordingly.

[0076] The test results of the process nodes will exist in various forms in the database, local disk or shared disk. For example, the test result information (test success or test failure) of the entire process node is stored in the database; when the data processing process is simple and the data volume is small, the test results may also be stored locally on the server where the data processing system is located. When the data processing process is complex and the data volume is large, for example, the entire data processing system will adopt a distributed deployment method, so that the test results may be stored on the shared disk of each server. Therefore, it is necessary to update the job object based on reading these stored test result information in order to complete the subsequent report generation steps.

[0077] Based on this, in the above steps S42 and S43, the created database object attributes are updated and assigned according to the information recorded in the database of each process node.

[0078] And according to the running test data saved by the data processing system running the data processing process, such as the time consumption information stored by the data processing system locally or in a shared disk and the information of each test point corresponding to the job object, the values ​​of the public attributes and checkpoint attributes in the created job object are updated.

[0079] For example, an example of a job object is as follows:

[0080] private String projected;

[0081] private String recordList;

[0082] private Jobtask jobtask;

[0083] private boolean isJobtaskPass=false;

[0084] private Boolean isNotTimeout=false;

[0085] private CommonValidation commonValidation;

[0086] private boolean isScan1HasBc=false;

[0087] private boolean isScan2HasBc=false;

[0088] private boolean isScan1HasPoslog=false;

[0089] private boolean isScan21HasPoslog=false;

[0090] private boolean isScan1PoslogNotEmpty=false;

[0091] private boolean isScan2PoslogNotEmpty=false;

[0092] The above-mentioned jobtask attributes represent its information in the database; CommonValidation is a public attribute, which represents the overall success or failure information of the data processing task corresponding to the job object, test situation statistics, etc.; of course, the job object also has the attribute information of the data processing task itself, as well as the checkpoint attributes contained in the data processing task.

[0093] The database can store the status information of data processing tasks, such as pending, processing, successful, etc. To change the status of a data processing task, you only need to change the corresponding field in the database, such as job status information. The database also records the start and end time of each data processing task.

[0094] Common attributes may include one or more of the following attribute information: the overall inspection results of all test points corresponding to the task inspection object; the total number of all test points corresponding to the entire task inspection object, the number of test points corresponding to the task inspection object that have run successfully, and the number of test points corresponding to the task inspection object that have failed to run, etc.

[0095] The overall inspection result of all test points corresponding to the task inspection object reflects a total test result of all test points corresponding to the task inspection object. For example, in some cases, it is preset that only when all test points are successfully executed, the overall inspection is considered to be passed. If the test result of one test point is fail, the overall inspection result of the task inspection object is not passed (or failed), or when the number of successfully executed test points is greater than the ratio of the sum of the number of all test points is greater than the preset ratio, the overall inspection result is considered to be passed, otherwise it is not passed, etc. The embodiment of the present invention does not limit the specific method of determining the overall inspection result.

[0096] An example of CommonValidation is as follows:

[0097] Public class CommonValidation{

[0098] private boolean isPass = true; / / Identifies the overall inspection result of the entire task;

[0099] private String validationResult; / / Identifies the entire task inspection information, such as how many test points were executed, the number of successes, and the number of failures;

[0100] private int total_testpoint; / / Identifies the total number of test points for the entire task;

[0101] private int total_success; / / Identifies the number of successful test points of the task;

[0102] private int total_fail; / / Indicates the number of failed test points of the task.

[0103] In the above step S41, when creating at least one task inspection object for each process node, the type of the task inspection object can be first determined according to the type of the process node to which it belongs; then, according to the number of business data files and business data folders corresponding to each process node in the data processing system, or according to the number of data processing results of the previous process node of the process node, the number of data processing tasks included in the process node is determined, and according to the determined number of data processing tasks, the number of corresponding task inspection objects is determined; finally, according to the determined type and number of task inspection objects, at least one task inspection object of the corresponding type and number is created.

[0104] The number of the above-mentioned task inspection objects can be determined according to the number of data processing tasks generated by each process node (module). Specifically, in a system where the data processing process is running, the business data (data data) involved in each process node may be divided into multiple files or multiple folders. Therefore, in an embodiment of the present invention, local data files can be used as the basis for judgment, that is, the number of files or folders into which the data is divided is equal to the number of data processing tasks opened in parallel. Similarly, the number of job objects that need to be created accordingly.

[0105] In order to support the result inspection of all data processing tasks of various types of data processing flows, different data processing systems of different data processing flows have different judgment criteria for how many data processing tasks each module has generated. In the embodiment of the present invention, the number of business data files or business data folders is used as the basis for judgment; it can also be determined by the number of data processing results output by the previous process node. Of course, other methods can also be used, which are not limited by the embodiment of the present invention.

[0106] For the inspection of data processing tasks, this solution believes that it is necessary to first determine the task status. The status of data processing tasks is mainly obtained from the database, that is, the information in the database is stored in the job object.

[0107] As mentioned above, the database can store the start and end times of each data processing task. In this way, for example, if the test report needs to count the execution time of each data processing task, this type of data needs to be calculated based on the start and end time values ​​of the task stored in the jobtask attribute of the job object.

[0108] For each type of data processing task, the test points (also known as checkpoints) to be checked are different. The check logic of this type of test point can be implemented through each customized job check service. For example, for task A, it needs to check whether the target file is generated on the shared disk (such as object storage OSS), whether the target file is in the correct format, etc.; for another example: for task B, it needs to check whether the data is stored in the database, whether the query interface has the expected data information, etc. The specific type of each test point corresponding to the task check object and the corresponding test results can be recorded through the checkpoint attributes of the task check object.

[0109] For a standardized test report, the test result information must be statistically analyzed, such as how many test points were run in total, how many succeeded, how many failed, etc. These can be obtained through the public properties of the corresponding job object.

[0110] The statistics of test results can be cleverly implemented through the Java reflection mechanism, that is, using the Java reflection mechanism to obtain the summary results of the test points (also checkpoints) in the job object.

[0111] The Java reflection mechanism is that in the running state, for any entity class, all the properties and methods of this class can be known; for any object, any of its methods and properties can be called; this function of dynamically obtaining information and dynamically calling object methods is called the reflection mechanism of the Java language.

[0112] Let's take a practical example to illustrate the reflection mechanism:

[0113]

[0114] For example, in the above code, the value in the Java class is obtained through the get method.

[0115] A specific process example is used to illustrate the process of creating, updating and generating test reports based on the above inspection objects. Figure 5 As shown, the process includes the following steps:

[0116] Step 1': Retrieve local job information. If successful, execute the following step 2'; otherwise, end the process.

[0117] The data processing system of the data processing flow queries the existing business data locally (such as a shared disk). For example, if the business data is divided into 10 business data folders, it can be considered that 10 data processing tasks (jobs) are generated.

[0118] Step 2', create a corresponding number of job objects;

[0119] Create a corresponding number of job objects based on the number in step 1.

[0120] Step 3', check the information of each data processing task in the database and update the job object accordingly; if the task is successfully executed, continue to step 4'; otherwise, if the task is successfully executed, jump directly to step 6';

[0121] Step 4': Continue to check the running status information, time consumption and other information of the successfully executed data processing tasks, and update the job object accordingly.

[0122] Step 5': Continue to update the job object according to the type of the task and the customized test check (test point) information of the task.

[0123] Step 6': Generate a corresponding test report based on the list of job objects.

[0124] The test report includes information about failed data processing tasks, successful data processing tasks, the time consumption of each successfully executed data processing task, details of the test points included, and the like.

[0125] In one embodiment, in the above step S21, based on the test result information in the task inspection object, statistics are collected on the full test results of the entire data processing process and / or the test results of each process node, which can be calculated by the data values ​​of the common attributes created by the job object, such as the aforementioned CommonValidation attribute.

[0126] In one embodiment, the above step S22 generates a corresponding test report, and the statistical results in step S21 can be displayed according to the formats and contents of various test reports preset by QA. Figure 6 and Figure 7 Example.

[0127] Reference Figure 6 As shown in the figure, the data segmentation report includes the total number of test points, the number of successful tests, the number of failed tests, etc. It also records the running status, time consumption, data verification results, etc. corresponding to each data group (data processing task).

[0128] Reference Figure 7 As shown, the total number of test points (also checkpoints) and the number of successes and failures among them, as well as the detailed content information and verification results of each test point are displayed.

[0129] Based on the same inventive concept, an embodiment of the present invention also provides an automatic generation device and a test server for a data processing flow test report. Since the principles of the problems solved by these devices and servers are similar to those of the aforementioned automatic generation method for a data processing flow test report, the implementation of the device and server can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0130] The embodiment of the present invention provides a device for automatically generating a data processing flow test report, referring to Figure 8 As shown, including:

[0131] The task check object generation module 81 is used to create at least one task check object corresponding to each process node in the data processing process, and after the test of each process node in the data processing process is completed, update the corresponding task check object using the test result information of each process node;

[0132] An acquisition module 82, used to acquire a list of at least one task inspection object corresponding to each process node in the data processing process;

[0133] The report generation module 83 is used to generate a test report of the data processing flow according to the test result information in the task inspection object.

[0134] In one embodiment, the above-mentioned report generation module 83 is specifically used to count the full test results of the entire data processing process and / or the test results of each process node according to the test result information in the task inspection object; and generate a corresponding test report based on the full test results of the entire data processing process and / or the test results of each process node obtained by statistics.

[0135] In one embodiment, the task check object generation module 81 creates at least one task check object of a corresponding type for each process node according to the type of the process node; the task check object includes at least one of the following: database result object attributes, common attributes and checkpoint attributes; after the process node test of the data processing process is completed, the created database result object attributes are updated and assigned according to the information recorded in the database of each process node; the created common attributes and checkpoint attributes are updated and assigned according to the running test data saved by the data processing system running the data processing process.

[0136] In one embodiment, the above-mentioned task inspection object generation module 81 determines the type of the task inspection object according to the type of the process node; determines the number of the task inspection objects according to the number of business data files and business data folders corresponding to each process node in the data processing system, or according to the number of data processing results of the previous process node of the process node; and creates at least one task inspection object of the corresponding type and the corresponding number according to the determined type and the number.

[0137] An embodiment of the present invention provides a test server, comprising: a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the instructions, when executed by the processor, can implement the above-mentioned method for automatically generating a data processing flow test report.

[0138] An embodiment of the present invention further provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implements the aforementioned method for automatically generating a data processing flow test report.

[0139] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0140] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0141] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0143] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for generating a data processing flow test report, It is characterized in that include: Creating at least one corresponding task check object for each process node in the data processing process, and after the test of each process node in the data processing process is completed, using the test result information of each process node to update the corresponding task check object; wherein the task check object is a Java object designed for each data processing task of the process node in the data processing process, and the task check object includes at least one of the following: database result object attributes, public attributes and checkpoint attributes; obtaining a list of at least one task check object corresponding to each process node in the data processing process; A test report of the data processing flow is generated based on the test result information recorded in the task inspection object.

2. The method according to claim 1, It is characterized in that Generating a test report of the data processing flow according to the test result information in the task inspection object, including: According to the test result information recorded in the task inspection object, the full test results of the entire data processing process and / or the test results of each process node are counted; Generate a corresponding test report based on the statistically obtained full test results of the entire data processing flow and / or the test results of each process node.

3. The method according to claim 1, It is characterized in that After the test of each process node of the data processing process is completed, the task inspection object is updated using the test result information of each process node, including: After the process node test of the data processing process is completed, the database result object attributes in the created task check object are updated and assigned according to the information recorded in the database for each process node; According to the running test data stored in the data processing system running the data processing flow, the common attributes and checkpoint attributes in the created task check object are updated and assigned values.

4. The method according to any one of claims 1 to 3, It is characterized in that At least one task inspection object is created for each process node in the data processing process, including: Determine the type of the task inspection object according to the type of the corresponding process node; Determine the number of data processing tasks included in the process node according to the number of business data files and business data folders corresponding to each process node in the data processing system, or according to the number of data processing results of the previous process node of the process node, and determine the number of corresponding task inspection objects according to the determined number of data processing tasks; According to the determined type and quantity of the task inspection object, at least one task inspection object of the corresponding type and the corresponding quantity is created.

5. The method according to claim 3, It is characterized in that The public attribute includes one or more of the following attribute information: the overall inspection result of all test points corresponding to the task inspection object; the total number of all test points corresponding to the entire task inspection object, the number of test points corresponding to the task inspection object that have run successfully, and the number of test points corresponding to the task inspection object that have failed to run; The checkpoint attributes include attribute information of the test point itself.

6. The method according to any one of claims 1 to 3, It is characterized in that The task check object is a Java object, and the test result information in the task check object is obtained from the Java object through a Java reflection mechanism.

7. An automatic generation device for data processing flow test report, It is characterized in that include: A task check object generation module is used to create at least one corresponding task check object for each process node in the data processing process, and after the test of each process node in the data processing process is completed, update the corresponding task check object using the test result information of each process node; wherein the task check object is a Java object designed for each data processing task of the process node in the data processing process, and the task check object includes at least one of the following: database result object attributes, public attributes and checkpoint attributes; An acquisition module, used to acquire a list of at least one task inspection object corresponding to each process node in the data processing process; A report generation module is used to generate a test report of the data processing flow according to the test result information recorded in the task inspection object.

8. The device according to claim 7, It is characterized in that The report generation module is specifically used to count the full test results of the entire data processing process and / or the test results of each process node according to the test result information recorded in the task inspection object; and generate a corresponding test report based on the full test results of the entire data processing process and / or the test results of each process node obtained by statistics.

9. A test server, It is characterized in that include: A memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the instructions, when executed by the processor, can implement the method for automatically generating a data processing flow test report as described in any one of claims 1-6.

10. A computer-readable storage medium having computer instructions stored thereon, It is characterized in that When the instruction is executed by the processor, the method for automatically generating a data processing flow test report as described in any one of claims 1 to 6 is implemented.