A script-based testing system, method and apparatus

Abstract

The application relates to the technical field of computing, and discloses a script-based test system, method and device. The system comprises a script management module, a trigger control module, a script execution module, a monitoring module and a report generation module. The script management module is used for storing each script file into a predetermined directory and storing script metadata and running parameters of each script file into a cache area. The trigger control module is used for acquiring target script metadata and target running parameters based on a target script identifier and generating a target execution instruction. The script execution module is used for acquiring a target script file according to the target execution instruction, repeatedly executing the target script file based on the target running parameters, and writing an execution result, a standard result and a process state into a temporary log. The monitoring module is used for generating each monitoring result based on the execution result, the standard result and the process state in the temporary log and storing the monitoring result into the cache area. The report generation module is used for acquiring a target execution result from the cache area and generating a test report based on the target execution result.

Description

Technical Field

[0001] This application relates to the field of computer script execution and testing, and in particular to a script-based testing system, method and device. Background Technology

[0002] In existing technologies, various scripting languages ​​such as Shell, Python, and JavaScript are widely used in software testing, data processing, and server maintenance to automate repetitive tasks. Current script execution methods mainly fall into two categories: manual terminal command input triggering and triggering via simple scheduled tasks.

[0003] For script execution monitoring, existing solutions mostly rely on manual viewing of terminal output logs or server log files; error troubleshooting requires technicians to analyze log content one by one to determine the error type and cause; after the script finishes running, the target data needs to be manually extracted from the output results, and server resource utilization needs to be manually recorded using additional monitoring tools (such as Top, Vmstat), and finally, a test report is manually written based on the above data.

[0004] The existing technology has the following drawbacks: First, the degree of automation is low. Simple scheduling tools can only achieve timed triggering and cannot meet complex requirements such as orderly execution of batch scripts and triggering based on preconditions, still requiring manual intervention. Second, the real-time monitoring is poor. Manual monitoring cannot track the script's running status synchronously throughout the process, easily missing the opportunity for errors to occur, and cannot collect server resource utilization data in real time. Third, the error handling efficiency is low. There is no automatic early warning mechanism after an error occurs, requiring manual discovery and investigation. Furthermore, log analysis relies on experience, and novices are prone to misjudgment, prolonging the problem-solving time. Fourth, the data and reports are not standardized. Manually collected data is prone to omissions and errors. Test reports written by different personnel have inconsistent formats, and key information is frequently missing, affecting the readability and usability of the reports.

[0005] Therefore, there is an urgent need for a script-based testing system to solve the problem that existing technologies cannot quickly and accurately complete tests and generate test reports in a standardized format. Summary of the Invention

[0006] In view of this, the present invention provides a script-based testing system, method and device, the main purpose of which is to solve the existing problems of being unable to complete tests quickly and accurately, and being unable to generate test reports in a uniform format.

[0007] To address the above problems, this application provides a script-based testing system, comprising: The script management module is used to store each script file to a predetermined directory and associate the script metadata of each script file with the corresponding running parameters and store them in a predetermined cache area. The script metadata includes at least: the storage path of the script file and the script identifier. The trigger control module is used to obtain target script metadata and target running parameters from the predetermined cache area based on the target script identifier, and generate target execution instructions; The script execution module is used to obtain the target script file from the predetermined directory according to the target execution instruction based on the predetermined script execution process, and to repeatedly execute the target script file based on the target running parameters, and to write the execution result, standard result and process status of each execution to a temporary log in real time. The monitoring module is used to generate monitoring results for each execution based on the execution results, standard results, and process status of each execution in the temporary log, and to store each monitoring result and the corresponding resource information in the cache area; The report generation module is used to obtain the target execution result that is found to be erroneous from the cache area, identify the target execution result based on each predetermined error record template, determine the error information, and generate a test report based at least on the error information; or obtain the target execution result that is found to be erroneous from the cache area, and generate a test report based on the erroneous target execution result.

[0008] Optionally, the trigger control module is specifically used for: Receive the target script identifier uploaded by the user; Based on the target script identifier, the cache region is located to determine the target script metadata containing the target script identifier; Obtain the target script metadata and the target runtime parameters associated with the target script metadata; The target running parameters include any one or more of the following parameters: number of concurrent executions, number of loop executions, runtime, and error rate threshold.

[0009] Optionally, the script execution module is specifically used for: Based on the storage path in the target script metadata of the target execution instruction and the target running parameters, the test tool command line is dynamically constructed; The script execution process is started based on a predetermined process startup module, and the target script file is retrieved from the predetermined directory according to the storage path in the test tool command line based on the script execution process, and the target script file is repeatedly executed according to the target running parameters in the test tool command line.

[0010] Optionally, the monitoring module is specifically used for: Create a status monitoring thread and a resource monitoring thread; Based on the status monitoring thread, the process status is queried from the cache area at a predetermined query time, and the execution results and standard results of each execution are obtained by parsing the temporary logs. Based on the execution results and standard results, the monitoring results and error rate of each execution are determined. Based on the resource thread, the server running the target script file is monitored for resources according to a predetermined monitoring time, and the resource information is obtained. The monitoring results, error rates, and corresponding resource information are stored in the cache area.

[0011] Optionally, the system further includes a configuration module, which is used to pre-configure error record templates corresponding to each error message, so as to build an error code library based on each error record template; each error record template includes, corresponding to the error message: error type identifier, error description keywords, regular expression, error cause description and suggested solution.

[0012] Optionally, the report generation module includes an error handling unit, which is used for: Extract the target field from the target execution result; The target field is matched with each error description keyword in each error record template to determine several candidate error record templates; Based on the regular expressions in each of the candidate error record templates, the target field is matched to determine the matching degree; The candidate error record template with the highest matching degree is used as the target error record template, so as to determine the target error information corresponding to the target field based on the target error record template.

[0013] Optionally, the error handling unit is further configured to: The current error rate is retrieved from the cache area at predetermined time intervals; A test stop command is generated based on the current error rate and the predetermined error rate threshold.

[0014] Optionally, the script execution module is further configured to: The performance test data from each execution is written to the temporary log, so that the status monitoring thread can read the performance test data from the temporary log and generate performance test metrics based on the performance test data.

[0015] To address the aforementioned problems, this application provides a script-based testing system method, comprising: Each script file is stored in a predetermined directory, and the script metadata and corresponding running parameters of each script file are associated and stored in a predetermined cache area. The script metadata includes at least: the storage path of the script file and the script identifier. Based on the target script identifier, the target script metadata and target runtime parameters are obtained from the predetermined cache area, and the target execution instructions are generated; Based on the predetermined script execution process, the target script file is obtained from the predetermined directory according to the target execution instruction, and the target script file is repeatedly executed based on the target running parameters. The execution results, standard results and process status of each execution are written to the temporary log in real time. Based on the execution results, standard results, and process status of each execution in the temporary log, a monitoring result for each execution is generated, and each monitoring result and the corresponding resource information are stored in the cache area; The system retrieves the target execution result from the cache area if the monitoring result is an error, identifies the target execution result based on each predetermined error record template, determines the error information, and generates a test report based at least on the error information; or it retrieves the target execution result from the cache area if the monitoring result is a correct execution, and generates a test report based on the correct target execution result.

[0016] To address the aforementioned problems, this application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program in the memory to implement the steps of the script-based testing method described above.

[0017] The script-based testing system, method, and device in this application store all scripts in a predetermined directory and associate the metadata of each script in a predetermined cache area. Subsequently, based on the target script identifier corresponding to the testing needs, the system can automatically retrieve the corresponding script metadata from the predetermined cache area, thereby automatically generating the target execution instructions. Then, based on the target execution instructions and the predetermined script execution process, the system can automatically and quickly perform load testing, thus completing the test quickly. At the same time, through the predetermined error recording template, the system can uniformly identify the monitoring results, thereby generating uniformly formatted error information and generating uniformly formatted test reports based on the error information.

[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a system block diagram of a script-based testing system according to an embodiment of this application; Figure 2 This is a block diagram of a test system in yet another embodiment of this application; Figure 3 A flowchart illustrating a script-based testing method according to another embodiment of this application; Figure 4 This is a structural block diagram of an electronic device according to another embodiment of this application. Detailed Implementation

[0020] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0021] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0022] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0023] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0024] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.

[0025] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0026] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely to serve as a representative basis for teaching those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0027] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0028] This application provides a script-based testing system, such as... Figure 1 As shown, it includes: Script management module 11 is used to store each script file to a predetermined directory and associate the script metadata of each script file with the corresponding running parameters and store them in a predetermined cache area. The script metadata includes at least: the storage path of the script file and the script identifier. In this embodiment, the testing system also includes an interaction module, which is used to respond to user interactions in a predetermined interaction area to obtain various script files. For example, the predetermined interaction area may be a script upload area and a parameter configuration area, and the interaction operations may include drag-and-drop, clicking, etc. That is, the user can upload various script files in the script upload area through clicks, drags, and other interactive operations, thereby enabling the testing system to obtain the script files. The system can then store each script file in a predetermined directory. Simultaneously, the user can perform parameter configuration operations in the parameter configuration area to configure the running parameters of the corresponding script files. These running parameters include: concurrency, number of loops, runtime, error rate threshold, etc.

[0029] After storing a script file in a predetermined directory, the script management module obtains the script file's storage path. Therefore, the storage path and script identifier can be used as script metadata, and the script metadata and runtime parameters can be associated and stored in a predetermined cache area. This allows the script metadata and runtime parameters to be automatically retrieved from the cache area when the script file is executed, facilitating the automatic generation of execution instructions and laying the foundation for rapid execution and testing of subsequent script files.

[0030] The trigger control module 12 is used to obtain target script metadata and target running parameters from the predetermined cache area based on the target script identifier, and generate target execution instructions; In this step, after all script files have been uploaded, the system determines which test object needs to be tested, thus identifying the script identifier corresponding to that object. Specifically, the interactive module can obtain the target script identifier manually entered in the designated script execution area. The trigger control module can then retrieve this target script identifier and, based on it, obtain the corresponding target script metadata and target execution parameters from the designated cache area. This allows for the rapid generation of the target execution instruction based on the target metadata and execution parameters. After generating the target execution instruction, the trigger control module can send it to the script execution module via a named pipe.

[0031] The script execution module 13 is used to obtain the target script file from the predetermined directory according to the target execution instruction based on the predetermined script execution process, and to repeatedly execute the target script file based on the target running parameters, and to write the execution result, standard result and process status of each execution to a temporary log in real time. In the specific implementation process, the script execution module can continuously listen to the named pipe to receive the target execution instruction, and dynamically construct the test tool command line according to the storage path and running parameters in the target execution instruction. The script execution process then retrieves the target script file from the predetermined directory according to the storage path in the test tool command line, and repeatedly executes the target script file according to the target running parameters in the test tool command line, thereby completing the automatic execution of the target script file, which helps to improve testing efficiency.

[0032] The monitoring module 14 is used to generate monitoring results for each execution based on the execution results, standard results and process status of each execution in the temporary log, and to store each monitoring result and the corresponding resource information in the cache area; In this embodiment, the monitoring module monitors temporary logs. Specifically, the monitoring module can pre-create two parallel sub-processes: a status monitoring process and a resource monitoring process. The status monitoring process generates monitoring results for each execution based on the execution results, standard results, and process status in the temporary logs. The resource monitoring process monitors server resources and obtains resource information corresponding to the monitoring results. After obtaining both the monitoring results and resource information, the monitoring module can store them in a cache area, facilitating the rapid and accurate generation of corresponding test reports based on the monitoring results and corresponding resource information.

[0033] The report generation module 15 is used to obtain the target execution result that is an execution error from the cache area, identify the target execution result based on each predetermined error record template, determine the error information, and generate a test report based at least on the error information, or obtain the target execution result that is a correct execution from the cache area and generate a test report based on the correct target execution result.

[0034] In this embodiment, the report generation module pre-stores error record templates. Therefore, the target execution result can be quickly identified using the error record templates, thereby quickly and accurately determining the error information, and then quickly and accurately generating a test report with a uniform format based on the error information.

[0035] In this embodiment, the system utilizes a script management module to uniformly store all scripts in a predetermined directory and associate the metadata of each script with a predetermined cache area. Subsequently, the trigger control module can automatically retrieve the corresponding script metadata from the predetermined cache area based on the target script identifier corresponding to the test requirements, thereby automatically generating the target execution instruction. Then, the script execution module can automatically and quickly perform load testing based on the target execution instruction and the predetermined script execution process, thus quickly completing the test. At the same time, through a predetermined error recording template, the monitoring results can be uniformly identified, thereby generating uniformly formatted error information. The report generation module can then generate a uniformly formatted test report based on the error information.

[0036] Another embodiment of this application provides a script-based testing system. In this embodiment, taking a software performance testing scenario as an example, the script file can be a JMeter script (.jmx format). Figure 2 As shown, the interactive system in this embodiment includes: The user interaction module includes a script upload area, a parameter configuration area, and a runtime control area. The script upload area responds to user upload requests for script files to retrieve the script files. The parameter configuration area responds to user parameter configuration requests for the script files to obtain the corresponding runtime parameters, including concurrency, loop count, runtime duration, and error rate threshold. The runtime control area responds to user interactions with the "Run" button to send the target script identifier / script ID to the trigger control module.

[0037] In this embodiment, the interaction module can also be configured with a result viewing area, which allows users to preview and download error logs and access test report download links.

[0038] The script management module is used to store each script file in a predetermined directory and associate the script metadata of each script file with the corresponding running parameters in a predetermined cache area. The script metadata includes at least the storage path of the script file and the script identifier.

[0039] In this embodiment, the script identifier includes a script ID, script name, and other numbers or strings used to uniquely identify the script. Specifically, the script management module receives script files uploaded by users. The script file format can be .jmx / .csv / .txt. It then stores the script file in a specified local directory and temporarily stores the script metadata and runtime parameters in a predetermined cache area, such as a Redis cache. The script ID is a randomly generated 8-character string that uniquely identifies the script. The script metadata includes: script ID, name, size, upload time, storage path, and a list of associated files. When writing the script metadata and runtime parameters to the Redis cache, the specific method is as follows: connect to the local Redis service (host=localhost, port=6379, password=custom password, db=0) using the redis-py library, call the set method, and write the script metadata and runtime parameters to the Redis cache.

[0040] The trigger control module receives the target script identifier uploaded by the user; searches the cache area based on the target script identifier to determine the target script metadata containing the target script identifier; and obtains the target script metadata and the target running parameters associated with the target script metadata; wherein, the target running parameters include any one or more of the following parameters: number of concurrent executions, number of loop executions, runtime, and error rate threshold.

[0041] In this embodiment, the trigger control module includes an HTTP interface for receiving the target script identifier / target script ID "script_id" (script ID) sent by the POST interface " / api / trigger / run". That is, after the user clicks the "Run" button on the interactive module, the interactive module sends a POST request carrying the script_id to this HTTP interface. The trigger control module connects to the local Redis service through the redis-py library and performs a query based on the target script identifier. The query statement is "script_${script_id}", and the query logic is: metadata = json.loads(r.get(f'script_{script_id}')). If the target script identifier does not exist, an error code 404 (script does not exist) is returned. If the target script identifier exists, the target script metadata, such as the script path, and the corresponding runtime parameters are extracted from the Redis cache. The trigger control module can then generate the target execution instruction based on the target script metadata and the corresponding runtime parameters.

[0042] After generating the target execution command, the trigger control module can send the execution command to the script execution module through a Windows local named pipe (NamedPipe, named "\\.\pipe\JMeterTriggerPipe"). The command format is a JSON string.

[0043] The script execution module is used to dynamically construct a test tool command line based on the storage path in the target script metadata and the target running parameters in the target execution instruction; to start the script execution process based on a predetermined process startup module; and to obtain the target script file from the predetermined directory based on the storage path in the test tool command line, and to repeatedly execute the target script file based on the target running parameters in the test tool command line.

[0044] In practical implementation, the script execution module continuously listens to the named pipe "\\.\pipe\JMeterTriggerPipe" to receive target execution instructions. Based on the storage path and target runtime parameters in the target execution instructions, it dynamically constructs a test tool command line. The subprocess module then starts the script execution process, which retrieves the target script file from the predetermined directory according to the storage path in the test tool command line and repeatedly executes the target script file according to the target runtime parameters in the test tool command line. Simultaneously, the script execution process captures standard output and error output in real time and writes them to a temporary log. After the process starts, it reports a "running" status; a normal termination reports "success"; and an abnormal termination reports "failure." In other words, the process status includes: running, success, and failure.

[0045] In this embodiment, the script execution module is further configured to: write the performance test data from each execution to the temporary log, so that the status monitoring thread can read the performance test data from the temporary log, generate performance test metrics based on the performance test data, and store the performance test metrics in a predetermined cache area. The performance test data includes: response time, throughput, number of successes, and number of errors. The performance test metrics include: average response time, 90% / 95% response time, throughput (TPS), success rate, and error rate.

[0046] The monitoring module is used to create a status monitoring thread and a resource monitoring thread; based on the status monitoring thread, it queries the process status from the cache area at a predetermined query time, and parses the temporary logs to obtain the execution results and standard results for each execution, and determines the monitoring results and error rate for each execution based on the execution results and standard results; based on the resource thread, it monitors the resources of the server running the target script file at a predetermined monitoring time to obtain the resource information; and stores each monitoring result, error rate, and corresponding resource information in the cache area.

[0047] In other words, the monitoring module includes a status monitoring unit and a resource utilization monitoring unit; The status monitoring unit is used to continuously parse the real-time output of the script execution process, extract request statistics (total number of requests, number of errors) line by line, dynamically calculate the real-time error rate, and store it in the Redis cache for the error handling module to read.

[0048] The resource utilization monitoring unit is used to: establish a TCP connection with the resource proxy service on the server under test, collect CPU utilization, memory usage, disk I / O rate, and network bandwidth usage at a frequency of 1 time / second, and store the raw data in Redis cache for the data processing module to retrieve.

[0049] In this embodiment, the monitoring module can pre-create two parallel sub-threads: a status monitoring thread and a resource monitoring thread. The status monitoring unit is used to query process status, parse logs, and calculate error rates based on predetermined query intervals. Simultaneously, the resource utilization monitoring unit is used by the resource monitoring thread to monitor server resources through the JMeter's built-in ServerAgent plugin to obtain resource information. This resource information includes any one or more of the following: CPU utilization, memory usage, disk I / O rate, and network bandwidth usage. In this embodiment, the method name for monitoring the process is status_monitor_task.

[0050] The report generation module includes an error handling unit and a generation unit. The error handling unit is used to: extract target fields from the target execution result; match the target fields with error description keywords in each of the error record templates to determine several candidate error record templates; match the target fields based on regular expressions in each of the candidate error record templates to determine the matching degree; and use the candidate error record template with the highest matching degree as the target error record template to determine the target error information corresponding to the target field based on the target error record template. The generation unit is used to: generate a test report based at least on the error information, or obtain monitoring results from the cache area that indicate the target execution result is correct, and generate a test report based on the correct target execution result. In this embodiment, since each error record template contains information such as error type identifier, error description keywords, regular expressions, and error reason descriptions, after determining the target error record template, the target error type identifier, target error description keywords, and target error reason descriptions in the target error record template can be used as the target error information of the target field, making error analysis of the target field faster and more accurate.

[0051] Specifically, the error handling unit is further configured to: obtain the current error rate from the cache area at predetermined time intervals; and generate a test stop instruction based on the current error rate and a predetermined error rate threshold. That is, the error handling unit can read the real-time error rate / current error rate from the Redis cache, then compare the real-time error rate / current error rate with a user-preset error rate threshold (default 50%). If the current error rate exceeds the error rate threshold, a stop instruction is generated and sent to the script execution module. Simultaneously, a report generation instruction is sent to the generation module to generate a complete error analysis report / test report.

[0052] In this embodiment, the system further includes a configuration module, which is used to pre-configure error record templates corresponding to each error message, and to build an error code library based on each error record template. Each error record template includes, corresponding to the error message, an error type identifier, error description keywords, a regular expression, an error cause description, and a suggested solution. In this embodiment, by pre-building the error code library, the error processing unit can quickly and accurately identify error messages in the target fields based on the templates in the error code library, thus ensuring the rapid generation of test reports based on the identified target error messages.

[0053] In this embodiment, each time the error handling unit captures an error request, it can extract the error information field, i.e., obtain the target field. Then, it performs a preliminary match between the target field and the error description keywords in the error record template to filter candidate error types. Next, it performs precise regular expression matching on each candidate item and outputs the identification result with the highest confidence. Finally, it writes the error type, cause, solution, and timestamp into the structured error log. In this embodiment, the error handling unit can read the real-time error rate from the Redis cache every second and compare it with a user-preset threshold, for example, a preset threshold of 50%. If the threshold is exceeded, a stop command is sent, and a complete error analysis report / test report is generated based on the generation unit. This report may include a list of error details and error type distribution statistics.

[0054] In this embodiment, the script-based testing system further includes a data processing module. This module is used to: extract average response time, 90% / 95% response time, throughput (TPS), success rate, error rate, and other metrics from the cache area / Redis cache; read resource utilization data from the Redis cache; remove outliers (data points deviating from the mean by 3 times the standard deviation) using the 3σ principle; complete missing timestamp data using linear interpolation; unify data units; perform deduplication; and store the processed data in JSON format in the Redis cache for use by the generation unit in the report generation module.

[0055] In this embodiment, the report generation module can generate a Word format test report based on python-docx 1.1.0 and reuse the command-line parameters of the testing tool to generate an HTML report. The Word report has two built-in templates: a normal completion template and an abnormal termination template. The report generation process is as follows: the generation unit obtains the structured error analysis results from the error handling unit, including: error type distribution statistics JSON, high-frequency aggregated alarm list, suggested solution entries, etc.; the generation unit stops selecting the corresponding template based on whether the current execution has triggered the error rate threshold; the generation unit reads the performance indicators output by the data processing module from the Redis cache, merges them with the error analysis results, and fills them into the corresponding information areas of the report according to the placeholder definitions. The information areas include: performance indicator area, resource utilization area, error analysis area, etc.; the generation unit synchronizes the HTML report to the server path and provides a preview link; the generation unit updates the download links of the Word and HTML reports to the Redis cache for users to download. In this embodiment, after the test is completed normally, a normal test report can be further generated based on the normal completion template. Specifically, the monitoring result obtained from the cache area can be the correct target execution result, and a test report can be generated based on the correct target execution result.

[0056] The system in this embodiment achieves automatic error classification and cause localization through a pre-built dedicated error code library and a keyword-regular expression combination algorithm, eliminating the need for manual line-by-line log analysis. In the control experiment (see specific implementation), the error localization time was reduced by an average of 80%. The monitoring module can track the script's running status in real time and dynamically calculate the error rate, with resource data collection accuracy no less than 1 time / second, ensuring data authenticity and completeness. Furthermore, the testing system in this application automates the entire process from script execution to report generation, requiring no manual intervention. Compared to purely manual operation, overall efficiency is improved by more than 60%. The test reports generated by this application are more standardized and regulated. Through automatic data collection and cleaning, error analysis results are structured and directly filled into the error analysis area of ​​the report after being output by the error processing unit, achieving standardized and unified test report format and fully presenting key information from the test. In addition, this application is highly flexible, with an expandable error code library and customizable report templates to adapt to different testing tools and business scenarios.

[0057] Another embodiment of this application provides a script-based testing method, such as... Figure 3 As shown, it includes: Step S101: Store each script file in a predetermined directory, and associate and store the script metadata and corresponding running parameters of each script file in a predetermined cache area. The script metadata includes at least: the storage path of the script file and the script identifier. Step S102: Obtain target script metadata and target running parameters from the predetermined cache area based on the target script identifier, and generate target execution instructions; Step S103: Based on the predetermined script execution process, obtain the target script file from the predetermined directory according to the target execution instruction, and repeatedly execute the target script file based on the target running parameters, and write the execution result, standard result and process status of each execution to the temporary log in real time. Step S104: Generate monitoring results for each execution based on the execution results, standard results, and process status of each execution in the temporary log, and store each monitoring result and the corresponding resource information in the cache area; Step S105: Obtain the target execution result that is an execution error from the cache area, identify the target execution result based on each predetermined error record template, determine the target error information, and generate a test report based at least on the error information; or obtain the target execution result that is a correct execution from the cache area, and generate a test report based on the correct target execution result.

[0058] In this embodiment, the step of obtaining target script metadata and target running parameters from the predetermined cache area based on the target script identifier and generating target execution instructions specifically includes: receiving the target script identifier uploaded by the user; searching the cache area based on the target script identifier to determine the target script metadata containing the target script identifier; obtaining the target script metadata and target running parameters associated with the target script metadata; wherein, the target running parameters include any one or more of the following parameters: concurrent execution count, number of loop executions, runtime, and error rate threshold.

[0059] In this embodiment, the process of retrieving the target script file from the predetermined directory based on the target execution instruction and repeatedly executing the target script file based on the target running parameters, according to the predetermined script execution process, specifically includes: dynamically constructing a test tool command line based on the storage path in the target script metadata and the target running parameters in the target execution instruction; starting the script execution process based on the predetermined process startup module, and retrieving the target script file from the predetermined directory based on the storage path in the test tool command line and repeatedly executing the target script file based on the target running parameters in the test tool command line.

[0060] In this embodiment, the process of generating monitoring results for each execution based on the execution results, standard results, and process status in the temporary log, and storing each monitoring result and corresponding resource information in the cache area, specifically includes: creating a status monitoring thread and a resource monitoring thread; querying the process status from the cache area according to a predetermined query time using the status monitoring thread, and parsing the temporary log to obtain the execution results and standard results for each execution, and determining the monitoring results and error rate for each execution based on the execution results and standard results; monitoring the resources of the server running the target script file according to a predetermined monitoring time using the resource thread, and obtaining the resource information; and storing each monitoring result, error rate, and corresponding resource information in the cache area.

[0061] In this embodiment, before determining the target error information, the script-based testing method further includes: pre-configuring error record templates corresponding to each error information to build an error code library based on each error record template; each error record template includes, corresponding to the error information: error type identifier, error description keywords, regular expression, error cause description, and suggested solution.

[0062] In this embodiment, the step of obtaining the monitoring result from the cache area as the target execution result of the execution error, and identifying the target execution result based on each predetermined error record template to determine the target error information, specifically includes: extracting the target field from the target execution result; matching the target field with each error description keyword in each of the error record templates to determine several candidate error record templates; matching the target field based on the regular expressions in each of the candidate error record templates to determine the matching degree; and using the candidate error record template with the highest matching degree as the target error record template to determine the target error information corresponding to the target field based on the target error record template.

[0063] In this embodiment, the script-based testing method further includes: obtaining the current error rate from the cache area at predetermined time intervals; and generating a test stop instruction based on the current error rate and a predetermined error rate threshold.

[0064] In this embodiment, during the specific implementation process, while writing the execution results, standard results, and process status of each execution to the temporary log in real time, the method further includes: writing the performance test data of each execution to the temporary log, so that the status monitoring thread can read the performance test data from the temporary log and generate performance test metrics based on the performance test data. The performance test data includes: response time, throughput, number of successes, and number of errors. The performance test metrics include: average response time, 90% / 95% response time, throughput (TPS), success rate, and error rate.

[0065] In this application, by uniformly storing each script in a predetermined directory and associating the metadata of each script in a predetermined cache area, the corresponding script metadata can be automatically retrieved from the predetermined cache area according to the target script identifier corresponding to the test needs, thereby automatically generating the target execution instruction. Subsequently, load testing can be automatically and quickly performed based on the target execution instruction and the predetermined script execution process, thus enabling rapid test completion. At the same time, through the predetermined error recording template, the monitoring results can be uniformly identified, thereby generating a uniformly formatted error message, and then generating a uniformly formatted test report based on the error message.

[0066] Another embodiment of this application provides an electronic device, such as... Figure 4 As shown, it includes at least a memory 1 and a processor 2. The memory 1 stores a computer program, and the processor 2 performs the following method steps when executing the computer program in the memory 1: Step 1: Store each script file in a predetermined directory, and associate and store the script metadata and corresponding running parameters of each script file in a predetermined cache area. The script metadata includes at least: the storage path of the script file and the script identifier. Step 2: Based on the target script identifier, obtain the target script metadata and target runtime parameters from the predetermined cache area, and generate the target execution instructions; Step 3: Based on the predetermined script execution process, obtain the target script file from the predetermined directory according to the target execution instruction, and repeatedly execute the target script file based on the target running parameters, and write the execution result, standard result and process status of each execution to the temporary log in real time; Step 4: Generate monitoring results for each execution based on the execution results, standard results, and process status in the temporary log, and store each monitoring result and the corresponding resource information in the cache area; Step 5: Obtain the target execution result that is an execution error from the cache area, and identify the target execution result based on each predetermined error record template to determine the error information, so as to generate a test report based at least on the error information, or obtain the target execution result that is a correct execution from the cache area and generate a test report based on the correct target execution result.

[0067] The specific implementation process of the above method steps can be found in the embodiments of any of the above script-based testing methods, and will not be repeated here.

[0068] In this application, by uniformly storing each script in a predetermined directory and associating the metadata of each script in a predetermined cache area, the corresponding script metadata can be automatically retrieved from the predetermined cache area according to the target script identifier corresponding to the test needs, thereby automatically generating the target execution instruction. Subsequently, load testing can be automatically and quickly performed based on the target execution instruction and the predetermined script execution process, thus enabling rapid test completion. At the same time, through the predetermined error recording template, the monitoring results can be uniformly identified, thereby generating a uniformly formatted error message, and then generating a uniformly formatted test report based on the error message.

[0069] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. Those skilled in the art can make various modifications or equivalent substitutions to this application within the scope and nature of this application, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A script-based testing system, characterized in that, include: The script management module is used to store each script file to a predetermined directory and associate the script metadata of each script file with the corresponding running parameters and store them in a predetermined cache area. The script metadata includes at least: the storage path of the script file and the script identifier. The trigger control module is used to obtain target script metadata and target running parameters from the predetermined cache area based on the target script identifier, and generate target execution instructions; The script execution module is used to obtain the target script file from the predetermined directory according to the target execution instruction based on the predetermined script execution process, and to repeatedly execute the target script file based on the target running parameters, and to write the execution result, standard result and process status of each execution to a temporary log in real time. The monitoring module is used to generate monitoring results for each execution based on the execution results, standard results, and process status of each execution in the temporary log, and to store each monitoring result and the corresponding resource information in the cache area; The report generation module is used to obtain the target execution result that is found to be erroneous from the cache area, identify the target execution result based on each predetermined error record template, determine the error information, and generate a test report based at least on the error information; or obtain the target execution result that is found to be erroneous from the cache area, and generate a test report based on the erroneous target execution result.

2. The system as described in claim 1, characterized in that, The trigger control module is specifically used for: Receive the target script identifier uploaded by the user; Based on the target script identifier, the cache region is located to determine the target script metadata containing the target script identifier; Obtain the target script metadata and the target runtime parameters associated with the target script metadata; The target running parameters include any one or more of the following parameters: number of concurrent executions, number of loop executions, runtime, and error rate threshold.

3. The system as described in claim 1, characterized in that, The script execution module is specifically used for: Based on the storage path in the target script metadata of the target execution instruction and the target running parameters, the test tool command line is dynamically constructed; The script execution process is started based on a predetermined process startup module, and the target script file is retrieved from the predetermined directory according to the storage path in the test tool command line based on the script execution process, and the target script file is repeatedly executed according to the target running parameters in the test tool command line.

4. The system as described in claim 1, characterized in that, The monitoring module is specifically used for: Create a status monitoring thread and a resource monitoring thread; Based on the status monitoring thread, the process status is queried from the cache area at a predetermined query time, and the execution results and standard results of each execution are obtained by parsing the temporary logs. Based on the execution results and standard results, the monitoring results and error rate of each execution are determined. Based on the resource thread, the server running the target script file is monitored for resources according to a predetermined monitoring time, and the resource information is obtained. The monitoring results, error rates, and corresponding resource information are stored in the cache area.

5. The system as described in claim 1, characterized in that, The system further includes a configuration module, which is used to pre-configure error record templates corresponding to each error message, so as to build an error code library based on each error record template; each error record template includes, corresponding to the error message: error type identifier, error description keywords, regular expression, error cause description and suggested solution.

6. The system as described in claim 5, characterized in that, The report generation module includes an error handling unit, which is used for: Extract the target field from the target execution result; The target field is matched with each error description keyword in each error record template to determine several candidate error record templates; Based on the regular expressions in each of the candidate error record templates, the target field is matched to determine the matching degree; The candidate error record template with the highest matching degree is used as the target error record template, so as to determine the target error information corresponding to the target field based on the target error record template.

7. The system as described in claim 6, characterized in that, The error handling unit is also used for: The current error rate is retrieved from the cache area at predetermined time intervals; A test stop command is generated based on the current error rate and the predetermined error rate threshold.

8. The system as described in claim 4, characterized in that, The script execution module is also used for: The performance test data from each execution is written to the temporary log, so that the status monitoring thread can read the performance test data from the temporary log and generate performance test metrics based on the performance test data.

9. A script-based testing method, characterized in that, include: Each script file is stored in a predetermined directory, and the script metadata and corresponding running parameters of each script file are associated and stored in a predetermined cache area. The script metadata includes at least: the storage path of the script file and the script identifier. Based on the target script identifier, the target script metadata and target runtime parameters are obtained from the predetermined cache area, and the target execution instructions are generated; Based on the predetermined script execution process, the target script file is obtained from the predetermined directory according to the target execution instruction, and the target script file is repeatedly executed based on the target running parameters. The execution results, standard results and process status of each execution are written to the temporary log in real time. Based on the execution results, standard results, and process status of each execution in the temporary log, a monitoring result for each execution is generated, and each monitoring result and the corresponding resource information are stored in the cache area; The system retrieves the target execution result from the cache area if the monitoring result is an error, identifies the target execution result based on each predetermined error record template, determines the error information, and generates a test report based at least on the error information; or it retrieves the target execution result from the cache area if the monitoring result is a correct execution, and generates a test report based on the correct target execution result.

10. An electronic device, characterized in that, It includes at least a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the script-based testing method of claim 9 when executing the computer program in the memory.

Images