Configurable defect and test case automatic generation method

Abstract

A configurable defect and test case automatic generation method is used in the technical field of software testing, historical error logs are clustered based on a K-means clustering algorithm, an error log type configuration table is generated, free configuration of whether the error logs need to be processed or not is achieved, automatic submission of defects is achieved, and the test efficiency is improved. The bottleneck and performance problems of the system are found in time and optimized; meanwhile, automatic generation of test cases is achieved, a test case library is further perfected, a Jenkins continuous integration tool is used for executing the automatic test cases at regular time, a test report is generated through Allure, the test efficiency and the controllability and stability of a tested system are improved, and meanwhile, in the process of continuously processing system error logs, the test efficiency is improved, and the test efficiency is improved. And self-improvement of the error log type library is realized.

Description

Technical Field

[0001] The present disclosure relates to the technical field of computer software testing, which belongs to the technical field of computer science, and further relates to a configurable defect and automated test case generation method, system, electronic device, and readable storage medium. Background Art

[0002] Computer software systems output a large number of error logs of various types every moment during operation. These error logs hide many potential system risks. The errors in them may be fatal errors to the system. If these errors are ignored or left unattended, they may even cause the computer software system to crash, resulting in irreversible consequences and losses. Therefore, it is necessary to process these error logs in the system in a timely manner.

[0003] When a problem occurs in the operation of a computer software system, checking the error logs in the system can quickly locate the cause of the problem, because these error logs usually contain enough information to identify, analyze and solve the problem. In addition, error logs can also be used to analyze the performance of computer software systems. By counting and analyzing the error logs of the system, system bottlenecks and performance problems can be found, so as to optimize them. At the same time, abnormal behaviors and potential problems in the system can be discovered in a timely manner. Therefore, timely perception and processing of error logs in computer software systems is crucial to ensure the stability of system operation.

[0004] However, some existing error log processing methods or anomaly detection methods generally process error logs in all systems, which is time-consuming and requires large processing load. There is no classification of the severity of error logs, and there is also a lack of problem experience database and accumulation of problem experience, and it is not possible to generate defect reports and test cases at the same time. This results in the inability to perform automated regression testing on related interfaces and functions after the problem is fixed, making it difficult to record the cause of the problem, the processing method and the result in a timely manner, and naturally it is impossible to share the experience of solving the problem.

[0005] Therefore, how to realize the configurability of error log processing; automatically generate defect reports for corresponding modules in real time to realize automatic submission of defects; promptly notify relevant personnel of new defect creation messages to promptly discover system bottlenecks and performance problems and optimize them, record and accumulate problem-solving materials in real time; update and improve the error log type library in real time; automatically generate test cases and improve the test case library; optimize the severity and priority of error types to improve software development efficiency and quality have become technical issues that need to be solved urgently. Summary of the invention

[0006] In response to the above technical problems, the present disclosure provides a configurable defect and automated test case generation method. Based on the K-means clustering algorithm, historical error logs are clustered to generate an error log type configuration table, which realizes the free configuration of whether the error log needs to be processed; the automatic submission of defects is realized, and the message of new defect creation is promptly notified to relevant personnel by push messages, which improves the perception of developers and testers on error logs, and timely discovers system bottlenecks and performance problems and optimizes them, thereby greatly reducing the scope of problem impact, reducing risks, and reducing losses. Developers can also record the cause of the defect and the solution in the defect in a timely manner, and archive the problem to the problem experience library in a timely manner, thereby continuously improving the problem experience library.

[0007] In response to the above technical problems, the present invention not only realizes the automatic generation of test cases, but also further improves the test case library, and uses the Jenkins continuous integration tool to execute automated test cases regularly, and uses Allure to generate test reports, which is convenient for timely regression testing and monitoring of relevant functions and interfaces, and sends the report details to relevant personnel in a timely manner through email and other means. Relevant personnel can understand the system operation status in a timely manner by viewing the case execution results, thereby improving the test efficiency and the controllability and stability of the tested system. In addition, in the process of continuously processing system error logs, the self-improvement of the error log type library is also achieved.

[0008] Specifically, in order to solve the above technical problems, the present disclosure provides a configurable defect and test case automatic generation method, system, electronic device and storage medium.

[0009] According to one aspect of the present disclosure, the present disclosure provides a configurable defect and test case automatic generation method, comprising the following steps:

[0010] (1) Data reading: read system log files in real time to obtain data to be processed;

[0011] (2) Interface information statistics: automatically obtain all interface information of the backend service interface layer, and write all the above interface information into the interface information table, thereby completing the interface information table;

[0012] (3) Error log type configuration: Use the K-means clustering algorithm to cluster historical error logs, and write the clustering results into the error log type configuration table;

[0013] (4) Error log analysis: The error log analysis module filters and analyzes the logs obtained in real time to obtain analysis data;

[0014] (5) Automatic defect submission: First, query the interface information table according to the interface name or path name obtained by parsing the error log to obtain the software module name, interface input parameters, software module development manager, and description information to which the error reporting interface belongs; if the information is not obtained, directly call the backend service interface to obtain the relevant information of the interface and write it into the interface information table; secondly, query the error log type configuration table to obtain the error type, status, severity level, and priority corresponding to the error log; thirdly, call the API interface of the defect management tool to automatically submit the defect of the module; finally, push the newly generated defect message to the relevant development manager or developer in a timely manner;

[0015] (6) Automatic generation of test cases: Based on the interface name or path name obtained from the error log analysis, query the interface information table to obtain the interface input parameter list, calling method, and software module information corresponding to the error reporting interface; based on the user data obtained from the error log analysis module, assign values ​​to the interface input parameters to generate the input data of the interface; call the test case generation interface to generate automated test cases, and write the relevant information into the test case library;

[0016] (7) Defect handling and archiving: The cause and solution of the defect should be recorded in the defect in a timely manner, and the problem should be archived in the problem experience database in a timely manner, so as to continuously improve the problem experience database;

[0017] (8) Scheduled automatic execution of test cases and test report generation: Use the Jenkins continuous integration tool to schedule all automated test cases, and use the Allure test report generation tool to generate test result reports.

[0018] Preferably, the interface information table includes the following fields: the interface information table mainly includes: interface call URL, interface name, interface input parameter list, calling method and module name.

[0019] Preferably, the error log type configuration table includes the following fields: error level, error type, error code, processing enablement status, severity level, processing priority, and remarks.

[0020] Preferably, the parsed data includes: error level, occurrence location, error type, IP address, timestamp, error path, exception information, user data, and interface name.

[0021] Preferably, the defect content includes: server IP address, occurrence time, exception information, error path, user data, error type, severity level, processing priority, error log processing status, whether it is a new error type (isNew), software module development person in charge, and environmental information.

[0022] Preferably, after the developer investigates the cause of the defect and solves the problem, the isNew field is used to determine whether it is a new error type. If it is determined to be a new error type and the error log does not need to be processed, the whether to enable in the configuration table is set to a closed state with one click, thereby achieving free control over whether the error log is processed;

[0023] Evaluate whether the severity level and processing priority of the error type are reasonable, modify the severity level and priority of the error type in the configuration table, so as to continuously optimize the urgency and accuracy of its processing.

[0024] Preferably, use the Jenkins continuous integration tool to execute all automated test cases of the system on a daily basis to monitor whether the system is running normally, or execute the corresponding automated test cases for a certain function to perform regression testing on related interfaces or functions.

[0025] Preferably, the error log type configuration module uses K-means clustering algorithm to cluster historical error logs, and writes the clustering results into the error log type configuration table, which mainly includes: error level, error type, error code, processing enablement status, severity level, and processing priority field. The severity level, processing priority, and whether to enable the processing switch are configured in the error log type configuration table. The enabling status in the error log type configuration table can control whether to process the corresponding type of error log, thereby realizing the configurability of error log processing. Since the K-means clustering algorithm is used for unsupervised learning, no pre-labeled data or training samples are required, so the error log type configuration module can automatically identify patterns and anomalies in error logs without manual intervention; and because the K-means clustering algorithm is easy to understand and implement, the error log type configuration module has a fast calculation speed, can process large-scale data sets, and can classify and analyze a large number of error logs in a short time; in addition, the K-means clustering algorithm has no special requirements for data types, so the error log type configuration module can be applied to continuous data, discrete data, and mixed data, and its clustering results have good interpretability, so that the error log type configuration module can intuitively show the similarities and differences between different error logs, which is very helpful for quickly locating and solving system problems. In summary, the error log type configuration module has the characteristics of high efficiency, flexibility and robustness, realizes the automatic configuration of error log types, and can write clustering results into the error log type configuration table in real time.

[0026] Preferably, after acquiring the system log in real time and filtering and parsing it, the error log type configuration module also queries the error log type configuration table according to the system error log type. If no relevant configuration information is found, it means that the system error log is a new system error log type. The new system error log type information will be written into the error log type configuration table, and the default state is enabled. Therefore, in the process of continuously processing error logs, unknown error log types will be continuously supplemented and improved to the configuration table, thereby realizing self-improvement of the error log type, obtaining detailed information of the error log in real time, and updating the error log type configuration table in real time.

[0027] Preferably, real-time analysis of the error log detailed information obtained in real time includes the following processing steps:

[0028] First, obtain real-time logs: Use Python to write scripts as log management tools to automatically obtain system logs in real time and aggregate them into the system log table. Visually display the obtained logs on the page to facilitate management personnel to view, and support offline log query.

[0029] Secondly, filter the acquired real-time logs: classify the error levels into error (ERROR), serious error (CRITICAL), fatal error (FATAL), debug information (DEBUG), standard information (INFO), and warning information (WARNING); use Python language to filter according to the error level to be processed, and only filter the logs with error levels of ERROR, CRITICAL, and FATAL. Do not process the logs of DEBUG, INFO, or WARNING temporarily; and configure the error level of the logs to be processed at the same time;

[0030] Finally, based on the filtered error log, obtain the data required for automatically submitting defects and generating automated test cases: the required data mainly includes: error level, occurrence location, error type, service IP address, occurrence time, error path, exception information, user data, interface name, and error code data.

[0031] Preferably, the automatic defect submission further includes determining whether the current error log needs to be processed, and the determination method includes the following steps:

[0032] (1) querying the error log type configuration table according to the log type in the log information obtained by the error log parsing module. If the log type is found but the type configuration state is disabled, the log is no longer processed;

[0033] (2) If the error log type configuration table can be found according to the log type, and the configuration status is enabled, the log needs to be processed to enter the process of automatically submitting defects and generating automated test cases;

[0034] (3) If no relevant configuration information is found by querying the error log type configuration table according to the log type, the error log type information is written into the error log type configuration table, and the status is enabled by default, and the severity level and priority are high by default; at the same time, the log continues to be processed and the process of automatically submitting defects and generating automated test cases is entered.

[0035] Preferably, the automatic defect submission further includes:

[0036] Query interface information: query the interface information table according to the interface name or path name obtained by parsing the error log, and obtain the name of the software module to which the error reporting interface belongs and the information of the person in charge of software module development; if not obtained, directly call the backend service interface to obtain the relevant information of the interface, and write the relevant information of the interface into the interface information table.

[0037] Preferably, the automatic defect submission further includes:

[0038] Call the new defect API interface of the defect management tool to generate defect content, which includes: server IP address, occurrence time, exception information, error path, user data, error type, error code, severity level, priority, error log processing status, whether it is a new error type (isNew), software module development person in charge, and environmental information.

[0039] Preferably, the automatic defect submission further includes:

[0040] Push newly generated defect messages to relevant development managers or developers in a timely manner.

[0041] According to another aspect of the present disclosure, the present disclosure provides a configurable defect and test case automatic generation system, comprising:

[0042] (1) Data reading module: reads system log files in real time to obtain data to be processed;

[0043] (2) Interface information statistics module: automatically obtains all interface information of the backend service interface layer, and writes all the above interface information into the interface information table, thereby completing the interface information table;

[0044] (3) Error log type configuration module: uses K-means clustering algorithm to cluster historical error logs and writes the clustering results into the error log type configuration table;

[0045] (4) Error log parsing module: The error log parsing module filters and parses the logs acquired in real time to obtain parsed data;

[0046] (5) Defect automatic submission module: First, according to the interface name or path name obtained by parsing the error log, query the interface information table to obtain the software module name, interface input parameters, software module development person in charge, and description information to which the error reporting interface belongs; if not obtained, directly call the backend service interface to obtain the relevant information of the interface and write it into the interface information table; secondly, query the error log type configuration table to obtain the error type, status, severity level and priority corresponding to the error log; thirdly, call the API interface of the defect management tool to automatically submit the defect of the module; finally, push the newly generated defect message to the relevant development person in charge or developer in a timely manner;

[0047] (6) Test case automatic generation module: query the interface information table according to the interface name or path name obtained by error log analysis, obtain the interface input parameter list, calling method, and software module information corresponding to the error reporting interface; assign values ​​to the interface input parameters according to the user data obtained in the error log analysis module to generate the input data of the interface; call the test case generation interface to generate an automated test case, and write its related information into the test case library;

[0048] (7) Defect handling and archiving module: The cause and solution of the defect are recorded in the defect in a timely manner, and the problem is archived in the problem experience database in a timely manner, so as to continuously improve the problem experience database;

[0049] (8) Test case scheduled automatic execution and test report generation module: Use the Jenkins continuous integration tool to schedule all automated test cases, and use the Allure test report generation tool to generate test result reports.

[0050] According to another aspect of the present disclosure, the present disclosure provides an electronic device, including:

[0051] a memory storing execution instructions; and

[0052] A processor executes the execution instructions stored in the memory, so that the processor executes the above method.

[0053] According to another aspect of the present disclosure, the present disclosure provides a readable storage medium, wherein the readable storage medium stores execution instructions, and the execution instructions are used to implement the above method when executed by a processor.

[0054] Beneficial effects of the present disclosure:

[0055] 1. Use the K-means clustering algorithm to cluster historical error logs, and store the clustering results in the error log type configuration table; at the same time, the configuration table will configure the bug level and whether to enable the processing switch to achieve configurable error log processing.

[0056] 2. Get error log information, filter and parse the log. Query the configuration table according to the error log type. If found and the status is enabled, parse the error log to obtain the error level, location, error type, IP address, timestamp, error path, exception information, user data, interface name, error code, etc., and obtain the corresponding severity level, processing priority, whether to enable, software module name and other information from the configuration table, call the API interface of the defect management tool, automatically generate a defect report for the corresponding module, and realize the automatic submission of defects.

[0057] 3. When the defect is successfully submitted in step 2, the new defect creation message will be promptly notified to relevant personnel by push messages. Relevant personnel can discover system problems and solve them in a timely manner, discover system bottlenecks and performance problems in a timely manner and optimize them, thereby greatly reducing the scope of problem impact, reducing risks, and reducing losses. The cause of the defect, solution, and plan are recorded in the defect in a timely manner, and problem solving information is accumulated.

[0058] 4. In step 2, if the configuration information of the error log is not found, it will be marked as a new error log type and a defect will be automatically submitted. At the same time, the error log type will be written into the configuration table and the status will be enabled by default, so as to continuously supplement and improve the unknown error log types and realize the self-improvement of the error log type library.

[0059] 5. Search the interface table according to the interface name obtained through interface resolution, obtain the input parameter list of the corresponding interface, and assign the user data in the error log to generate the input data of the interface. Then call the test case generation interface implemented in Python to generate automated test cases for the corresponding interface, thereby realizing the automatic generation of test cases and further improving the test case library.

[0060] 6. After the developer handles the problem, the new error type field is used to determine if it is a new error type and does not need to be processed. The enablement in the configuration table can be set to off with one click, thereby achieving free control over error log processing. In addition, the developer can also evaluate whether the severity level and priority of the error type are reasonable. The urgency and accuracy of its processing can be continuously optimized by modifying the severity level and priority of the error type in the configuration table. At the same time, the developer needs to archive the problem to the problem library in a timely manner. The problem experience library provides developers with an effective basis for problem handling, which can improve the speed at which developers solve problems and avoid similar problems in future development, thereby improving development quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings illustrate exemplary embodiments of the present disclosure and are used together with the description thereof to explain the principles of the present disclosure, wherein these drawings are included to provide a further understanding of the present disclosure, and the drawings are included in and constitute a part of this specification. By describing the embodiments of the present disclosure in detail in conjunction with the accompanying drawings, the above and other purposes, features, and advantages of the present disclosure will become more apparent.

[0062] Figure 1 Shows the overall architecture diagram

[0063] Figure 2 shows the interface information data preparation diagram

[0064] Figure 3 Figure 1 shows the error log type configuration data preparation diagram

[0065] Figure 4 A self-improvement flowchart showing the error log type configuration table

[0066] Figure 5 Shows the log parsing flow chart

[0067] Figure 6 Shows the defect automatic submission flow chart

[0068] Figure 7 Shows the test case automatic generation flow chart

[0069] Figure 8 Defect handling flow chart

[0070] Fig. 9 The overall process flow chart is shown DETAILED DESCRIPTION

[0071] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0072] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0073] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0074] The terms used in the present disclosure are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups are not excluded from existence or addition. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0075] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present application. In this specification, the schematic representations of the above terms are not necessarily the same embodiment / method or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments / methods or examples described in this specification and the features of the different embodiments / methods or examples, without contradiction.

[0076] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0077] The present invention provides a configurable defect and automated test case generation method, which can freely configure error log types, automatically submit defects and generate test cases, and automatically improve the error log type library, test case library and problem experience library, which can improve the test efficiency and system development quality, thereby improving the controllability and stability of the system under test.

[0078] The overall processing architecture of the configurable defect and automated test case generation method is as follows: Figure 1 As shown, it mainly includes: underlying database layer, tool layer, functional module layer and system log file. Among them, the underlying database layer includes interface information table, error log type configuration table, problem experience library and test case library; the tool layer includes defect management tool, Jenkins continuous integration tool and Allure test report tool; the functional module layer includes interface information statistics module, error log type configuration module, error log parsing module, defect automatic submission module, test case automatic generation module, defect handling and archiving module, test case scheduled automatic execution and test report generation module.

[0079] The entire process of the configurable defect and automated test case generation method includes the collaborative processing of the following seven modules: interface information statistics module, error log type configuration module, error log parsing module, defect automatic submission module, test case automatic generation module, defect processing and archiving module, test case scheduled automatic execution and test report generation module.

[0080] 1. Interface information statistics module

[0081] The processing flow of the interface information statistics module is as follows: Figure 2 The method mainly includes: using Python to write code to automatically obtain all interface information of the backend service interface layer on a regular basis, and writing interface information that is not in the interface information table into the interface information table, thereby continuously improving the interface information table. The interface information table mainly includes: interface call URL, interface name, interface input parameter list, calling method, software module name, software module development manager and other fields, in preparation for the subsequent automatic submission of defects and test cases.

[0082] 2. Error log type configuration module

[0083] like Figure 3 As shown, the error log type configuration module uses the K-means clustering algorithm to cluster historical error logs, and writes the clustering results into the error log type configuration table, which mainly includes fields such as error level, error type, error code, processing enablement status, severity level, and processing priority. The severity level, processing priority, and whether to enable the processing switch are configured in the error log type configuration table. The enabling status in the error log type configuration table can control whether to process the corresponding type of error log, thereby realizing the configurability of error log processing. Since the K-means clustering algorithm is used for unsupervised learning, no pre-labeled data or training samples are required, so the error log type configuration module can automatically identify patterns and anomalies in error logs without manual intervention; and because the K-means clustering algorithm is easy to understand and implement, the error log type configuration module has a fast calculation speed, can process large-scale data sets, and can classify and analyze a large number of error logs in a short time; in addition, the K-means clustering algorithm has no special requirements for data types, so the error log type configuration module can be applied to continuous data, discrete data, and mixed data, and its clustering results have good interpretability, so that the error log type configuration module can intuitively show the similarities and differences between different error logs, which is very helpful for quickly locating and solving system problems. In summary, the error log type configuration module has the characteristics of high efficiency, flexibility and robustness, realizes the automatic configuration of error log types, and can write clustering results into the error log type configuration table in real time.

[0084] Furthermore, if Figure 4 As shown, after obtaining the system log in real time and filtering and parsing it, the error log type configuration module also queries the error log type configuration table according to the system error log type. If no relevant configuration information is found, it means that the system error log is a new system error log type. The new system error log type information will be written into the error log type configuration table, and the default state is enabled. Therefore, in the process of continuously processing the error log, unknown error log types will be continuously supplemented and improved to the configuration table, thereby realizing the self-improvement of the error log type, obtaining detailed information of the error log in real time, and updating the error log type configuration table in real time.

[0085] 3. Error log parsing module

[0086] like Figure 5 As shown, the error log parsing module performs real-time parsing of the error log detailed information obtained in real time, including the following processing steps:

[0087] First, obtain real-time logs. Use Python language to write scripts as log management tools to automatically obtain system logs in real time and aggregate them into the system log table. Visually display the obtained logs on the page to facilitate management personnel to view, and support offline log query.

[0088] Secondly, filter the acquired real-time logs. The error levels are divided into multiple levels such as error (ERROR), serious error (CRITICAL), fatal error (FATAL), debug information (DEBUG), standard information (INFO), and warning information (WARNING).

[0089] Use Python language to filter according to the error level to be processed, only filter the logs with error levels of ERROR, CRITICAL and FATAL, and do not process DEBUG, INFO or WARNING logs for the time being. At the same time, the logs with what error level should be processed can also be configured.

[0090] Finally, based on the filtered error log, the data required for automatically submitting defects and generating automated test cases is obtained, mainly including: error level, occurrence location, error type, service IP address, occurrence time, error path, exception information, user data, interface name, error code and other data.

[0091] 4. Defect automatic submission module

[0092] like Figure 6 As shown, the defect automatic submission module includes the following processing steps:

[0093] First, determine whether the current error log needs to be processed. The determination method is as follows:

[0094] (1) The error log type configuration table is queried according to the log type in the log information obtained by the error log parsing module. If the log type is found but the type configuration state is disabled, the log is no longer processed.

[0095] (2) If the error log type configuration table can be found according to the log type, and the configuration status is enabled, the log needs to be processed to enter the process of automatically submitting defects and generating automated test cases.

[0096] (3) If no relevant configuration information is found by querying the error log type configuration table according to the log type, the error log type information is written into the error log type configuration table, and the state is enabled by default, and the severity level and priority are high by default. At the same time, the log is processed continuously to enter the process of automatically submitting defects and generating automated test cases.

[0097] Secondly, query the interface information. According to the interface name or path name obtained by parsing the error log, query the interface information table to obtain the name of the software module to which the error interface belongs, the person in charge of the software module development, and other information; if not obtained, directly call the backend service interface to obtain the relevant information of the interface, and write the relevant information of the interface into the interface information table.

[0098] Thirdly, the new defect API interface of the defect management tool is called to generate defect content, which includes: server IP address, occurrence time, exception information, error path, user data, error type, error code, severity level, priority, error log processing status, whether it is a new error type (isNew), software module development person in charge, environment, etc. The defect management tool is enterprise-level R&D management tool and solution software such as Swimlane, ONES or JIRA.

[0099] Finally, the newly generated defect message is pushed to the relevant development manager or developer in a timely manner. After receiving the message, the development manager or developer judges the severity and scope of the problem based on the problem level and description, and takes timely measures to control risks, thereby improving the development and testing awareness of error logs, and can also promptly discover system bottlenecks and performance problems and optimize them, which greatly reduces the scope of problem impact, reduces risks, and reduces losses.

[0100] 5. Test case automatic generation module

[0101] like Figure 7 As shown, the test case automatic generation module includes the following processing steps:

[0102] 1. Obtain the data required for the automated test case generation template. The content of the case generation template includes: test environment IP address, table header, interface call URL, request method, interface input parameter JSON string, expected status code, expected return content, function description, etc. You can set different automated test case generation template contents according to specific business functions.

[0103] (1) Obtain the required interface data under test according to the template generated by the automated test case. According to the interface name or path name obtained by parsing the error log, query the interface information table to obtain the header, interface call URL, request method, interface input parameter list, and software module corresponding to the error reporting interface, and store this data in the test case data table.

[0104] (2) Obtain the input data of the interface under test of the automated test case template. According to the interface name or path name obtained by parsing the error log, query the interface information table, obtain the error interface input parameter list, assign the user data obtained by the log parsing module as the interface input parameter, generate the input data of the interface, and store the input data in the test case data table according to the interface name.

[0105] 2. Use data processing tools to copy the original test data set in the test case data table to avoid overwriting the original test data set when the case is generated.

[0106] 3. Use Python to implement the public interface method case_generate_common() to generate the automated test case test_case.py for the interface under test, and add the command: pytest.main(['-vs','--capture=sys','--html=.. / Outputs / reports / pytest.html','--alluredir=.. / Outputs / allure']) in the test_case.py test case. After executing the test case, the execution result report pytest.html will be automatically generated in the path " / Outputs / reports" for subsequent analysis and troubleshooting.

[0107] 4. Analyze execution results: Generate a test report based on the test case execution results. The report content includes pass rate, failed cases, error details, etc. Analyze and locate problems in the automated test case code, and optimize the case code to meet test requirements.

[0108] 5. Expand more data sets by modifying the original data set. Use the fuzz testing tool AFL (American Fuzzy Lop) to mutate the test cases. The main mutation types used are Arithmetic (integer addition / subtraction arithmetic operations), Interest (special content replacement) and Splice (joining two files to get a new file). Mutate the original test data set to generate a new test case set. You can also perform batch mutation on the specified test data set according to the mutation operator and multiple test data set paths, and generate new test case sets in batches. These new test data sets are used to test the behavior of the interface in the face of abnormal or boundary conditions, thereby further supplementing the test scenarios of the interface, more comprehensively covering boundary conditions and special cases, and improving test coverage.

[0109] 6. All the generated automated test cases are written into the test case library, thereby further improving the test case library.

[0110] In addition, the Jenkins continuous integration tool is used to execute all automated test cases of the system on a daily basis to monitor whether the system is running normally, or to execute corresponding automated test cases for a certain function, and to perform regression testing on related interfaces or functions, thereby improving test efficiency and system stability, especially for agile development projects.

[0111] 6. Defect handling and archiving module

[0112] like Figure 8 As shown, after the developer investigates the cause of the defect and solves the problem, the isNew field is used to determine whether it is a new error type. If it is determined to be a new error type and it is evaluated that the error log does not need to be processed, the enable state in the configuration table can be set to disabled with one click, thereby achieving free control over whether the error log is processed.

[0113] In addition, developers can also evaluate whether the severity level and priority of the error type are reasonable, and can modify the severity level and priority of the error type in the configuration table to continuously optimize the urgency and accuracy of its processing.

[0114] Developers can also record the causes and solutions of defects in a timely manner, further accumulate problem-solving experience, and archive problems in a problem experience library in a timely manner, thereby continuously improving the problem experience library and improving the quality of system development.

[0115] 7. Test case timing automatic execution and test report generation module

[0116] Use Jenkins continuous integration tool to execute all automated test cases regularly, use Allure test report generation tool to generate test result report, and send the report details to relevant personnel via email, so that relevant personnel can understand the execution status of test cases in time, understand and monitor the operation status of the entire system in time, and improve the stability of the system under test. The overall processing flow chart is as follows Fig. 9 shown.

[0117] The present disclosure realizes the free configuration of system error log processing, and realizes the self-improvement of error log types in the process of continuous error log processing. It realizes the automatic submission of defects, and timely notifies the development and test personnel of new defect creation messages by push messages, improves the perception of development and testing to error logs, and can timely discover the bottlenecks and performance problems of the system and optimize them, thereby greatly reducing the scope of impact, reducing risks, and reducing losses. At the same time, developers can also record the causes and solutions of problems in defects in a timely manner, and archive problems to the problem experience library in a timely manner, thereby continuously improving the problem experience library, and providing a strong basis for development to solve problems in a timely manner. In addition, the automatic generation of test cases is realized, the test case library is further improved, and Jenkins is used to execute automated test cases regularly, and automatic regression testing and monitoring of functions and interfaces are performed, and Allure is used to generate test result reports, which improves the test efficiency and the stability of the system under test.

[0118] Any process or method description in the flowchart of the present disclosure or described in other ways herein can be understood as a module, fragment or part of a code representing one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes other implementations, in which the functions may not be performed in the order shown or discussed, including performing the functions in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the technical field to which the embodiments of the present disclosure belong. The processor performs the various methods and processes described above. For example, the method implementation in the present disclosure can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).

[0119] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions on an instruction execution system, apparatus, or device), or in combination with such instruction execution systems, apparatuses, or devices.

[0120] For the purposes of this specification, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, a device on the Internet, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting or, if necessary, processing in another suitable manner, and then stored in a memory.

[0121] It should be understood that the various parts of the present disclosure can be implemented with hardware, software or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented with software stored in a memory and executed by a suitable instruction execution system. For example, if implemented with hardware, as in another embodiment, it can be implemented with any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0122] A person of ordinary skill in the art of the present disclosure may understand that all or part of the steps of implementing the above-mentioned implementation method may be completed by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the implementation method.

[0123] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a readable storage medium. The storage medium may be a read-only memory, a disk or an optical disk, etc.

[0124] The present disclosure also provides an electronic device, including: a memory, the memory storing execution instructions; and a processor or other hardware module, the processor or other hardware module executes the execution instructions stored in the memory, so that the processor or other hardware module executes the above method.

[0125] The present disclosure also provides a readable storage medium, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the above method.

[0126] In summary, the method disclosed in the present invention has the following advantages:

[0127] 1. Use K-means clustering algorithm to cluster historical error logs, and store the clustering results in the error log type configuration table; at the same time, the configuration table will configure the defect severity level, priority and whether to enable the processing switch to achieve configurable error log processing.

[0128] 2. Obtain error log information in real time, query the error log type configuration table according to the error log type, and if found and the status is enabled, parse the error log to obtain the IP address, occurrence time, exception information, path list, user data, and the severity level, processing priority, error log processing status, software module name and other information of this type, call the API interface of the defect management tool, automatically generate defects for the corresponding software module, and realize automatic submission of defects.

[0129] 3. When the defect is successfully submitted in step 2, the new defect creation message will be promptly notified to relevant personnel by push messages. Relevant personnel can discover system problems and solve them in a timely manner, discover system bottlenecks and performance problems in a timely manner and optimize them, thereby greatly reducing the scope of problem impact, reducing risks, and reducing losses. The cause of the defect, solution, and plan are recorded in the defect in a timely manner, and problem solving information is accumulated.

[0130] 4. In step 2, if the configuration information of the error log is not found, it will be marked as a new error log type and a defect will be automatically submitted. At the same time, the error log type will be written into the error log type configuration table, and the status will be enabled by default, so as to continuously supplement and improve the unknown error log types and realize the self-improvement of the error log type library.

[0131] 5. Parse the error log information, obtain the list of error interface names, search the interface table according to the interface name, obtain the input parameter list of the corresponding interface, and assign the user data in the error log as the interface input parameter to generate the input data of the interface, and then call the test case generation interface implemented in Python to generate automated test cases for the corresponding interface, thereby realizing the automatic generation of test cases and further improving the test case library.

[0132] 6. After the developer handles the problem, the new error type field is used to determine if it is a new error type and does not need to be processed. The enablement in the configuration table can be set to off with one click, thereby achieving free control over error log processing. In addition, the developer can also evaluate whether the severity level and priority of the error type are reasonable. The urgency and accuracy of its processing can be continuously optimized by modifying the severity level and priority of the error type in the configuration table. At the same time, the developer needs to archive the problem to the problem library in a timely manner. The problem experience library provides developers with an effective basis for problem handling, which can improve the speed at which developers solve problems and avoid similar problems in future development, thereby improving development quality.

[0133] It can be seen that the present disclosure has achieved good results in actual use, and can realize timely, effective and accurate allocation of hot issues involved in the hotline case text data information to the corresponding social function management departments, and realize accurate and refined allocation of these special text data to social function management departments.

[0134] So far, the technical solutions of the present disclosure have been described in conjunction with the preferred implementation methods shown in the accompanying drawings. However, those skilled in the art should understand that the above implementations are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. The protection scope of the present disclosure is obviously not limited to these specific implementations. Without departing from the principles of the present disclosure, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions are still within the scope of the present disclosure.

Claims

1. A configurable defect and test case automatic generation method, characterized in that: The steps include: (1) Data reading: read system log files in real time to obtain data to be processed; (2) Interface information statistics: automatically obtain all interface information of the backend service interface layer, and write all the above interface information into the interface information table, thereby completing the interface information table; (3) Error log type configuration: Use the K-means clustering algorithm to cluster historical error logs, and write the clustering results into the error log type configuration table; (4) Error log analysis: The error log analysis module filters and analyzes the logs obtained in real time to obtain analysis data; (5) Automatic defect submission: First, query the interface information table according to the interface name or path name obtained by parsing the error log to obtain the software module name, interface input parameters, software module development manager information, and description information to which the error reporting interface belongs; if the information is not obtained, directly call the backend service interface to obtain the relevant information of the interface and write it into the interface information table; secondly, query the error log type configuration table to obtain the error type, status, severity level, and priority level corresponding to the error log; thirdly, call the API interface of the defect management tool to automatically submit the defect of the module; finally, push the newly generated defect message to the relevant development manager or developer in a timely manner; (6) Automatic generation of test cases: Based on the interface name or path name obtained from the error log analysis, query the interface information table to obtain the interface input parameter list, calling method, and software module information corresponding to the error reporting interface; based on the user data obtained from the error log analysis module, assign values ​​to the interface input parameters to generate the input data of the interface; call the test case generation interface to generate automated test cases, and write the relevant information into the test case library; (7) Defect handling and archiving: The cause and solution of the defect should be recorded in the defect in a timely manner, and the problem should be archived in the problem experience database in a timely manner, so as to continuously improve the problem experience database; (8) Scheduled automatic execution of test cases and test report generation: Use the Jenkins continuous integration tool to schedule all automated test cases, and use the Allure test report generation tool to generate test result reports.

2. A configurable defect and test case automatic generation method according to claim 1, characterized in that: The interface information table includes the following fields: interface call URL, interface name, interface input parameter list, calling method and module name.

3. A configurable defect and test case automatic generation method according to claim 1 or 2, characterized in that: The error log type configuration table includes the following fields: error level, error type, error code, processing enablement status, severity level, processing priority, and remarks.

4. A configurable defect and test case automatic generation method according to claim 1, 2 or 3, characterized in that: The parsed data includes: error level, occurrence location, error type, IP address, timestamp, error path, exception information, user data, and interface name.

5. A configurable defect and test case automatic generation method according to claim 1 or 2, characterized in that: The defect content includes: server IP address, occurrence time, exception information, error path, user data, error type, severity level, processing priority, error log processing status, whether it is a new error type (isNew), software module development person in charge information, and environmental information.

6. A configurable defect and test case automatic generation method according to claim 1, characterized in that: After the developer investigates the cause of the defect and solves the problem, the isNew field is used to determine whether it is a new error type. If it is a new error type and the error log does not need to be processed, the whether to enable in the configuration table is set to a closed state, thereby achieving free control over whether the error log is processed; Evaluate whether the severity level and processing priority of the error type are reasonable, modify the severity level and priority of the error type in the configuration table, and continuously optimize the urgency and accuracy of the processing.

7. A configurable defect and test case automatic generation method according to claim 1, characterized in that: Use the Jenkins continuous integration tool to execute all automated test cases of the system on a daily basis to monitor whether the system is running normally, or execute corresponding automated test cases for a certain function, and perform regression testing on related interfaces or functions.

8. A configurable defect and test case automatic generation system, characterized in that: include: (1) Data reading module: reads system log files in real time to obtain data to be processed; (2) Interface information statistics module: automatically obtains all interface information of the backend service interface layer, and writes all the above interface information into the interface information table, thereby completing the interface information table; (3) Error log type configuration module: uses K-means clustering algorithm to cluster historical error logs and writes the clustering results into the error log type configuration table; (4) Error log parsing module: The error log parsing module filters and parses the logs obtained in real time to obtain parsed data; (5) Defect automatic submission module: First, according to the interface name or path name obtained by parsing the error log, query the interface information table to obtain the software module name, interface input parameters, software module development person in charge information, and description information to which the error reporting interface belongs; if not obtained, directly call the backend service interface to obtain the relevant information of the interface and write it into the interface information table; secondly, query the error log type configuration table to obtain the error type, status, severity level and priority corresponding to the error log; thirdly, call the API interface of the defect management tool to automatically submit the defect of the module; finally, push the newly generated defect message to the relevant development person in charge or developer in a timely manner; (6) Test case automatic generation module: query the interface information table according to the interface name or path name obtained by error log analysis, obtain the interface input parameter list, calling method, and software module information corresponding to the error reporting interface; assign values ​​to the interface input parameters according to the user data obtained in the error log analysis module to generate the input data of the interface; call the test case generation interface to generate an automated test case, and write its related information into the test case library; (7) Defect handling and archiving module: The cause and solution of the defect are recorded in the defect in a timely manner, and the problem is archived in the problem experience database in a timely manner, so as to continuously improve the problem experience database; (8) Test case scheduled automatic execution and test report generation module: Use the Jenkins continuous integration tool to schedule all automated test cases, and use the Allure test report generation tool to generate test result reports.

9. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instruction stored in the memory so that the processor executes the method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images