Script quality evaluation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a script quality evaluation method and device, electronic equipment and a storage medium, and the method comprises the steps: testing a target software function containing positioning instrumentation by running a to-be-evaluated test script, and outputting an instrumentation log through the positioning instrumentation; determining a multi-dimensional quality index of the to-be-evaluated test script based on the instrumentation log; and performing quality evaluation on the to-be-evaluated test script based on the multi-dimensional quality index. According to the embodiment of the invention, the evaluation accuracy when the test script is evaluated can be ensured, and the evaluation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a script quality assessment method, device, electronic device and storage medium. Background Art

[0002] Automated testing can be used to reduce testing costs and improve efficiency when testing software functions. Automated testing allows computers to automatically execute test steps and verify test results based on test scripts. To ensure test reliability, the quality of test scripts must be ensured, necessitating accurate assessment of their quality. Existing test scripts are primarily evaluated manually, relying on the subjective experience of evaluators, which can affect the accuracy of evaluation results and reduce efficiency. Summary of the Invention

[0003] The embodiments of the present invention provide a script quality assessment method, device, electronic device and storage medium, which can ensure the assessment accuracy when evaluating test scripts and improve the assessment efficiency.

[0004] In a first aspect, an embodiment of the present invention provides a script quality assessment method, comprising:

[0005] Test the target software function containing the positioning instrumentation by running the test script to be evaluated, and output the instrumentation log through the positioning instrumentation;

[0006] Determining multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log; and

[0007] Performing a quality assessment on the test script to be assessed based on the multi-dimensional quality indicators.

[0008] In a second aspect, an embodiment of the present invention provides a script quality assessment device, comprising:

[0009] A test running and log acquisition module, configured to test target software functions containing positioning instrumentation by running a test script to be evaluated, and output instrumentation logs through the positioning instrumentation;

[0010] a quality indicator acquisition module, configured to determine multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log; and

[0011] A quality assessment module is used to perform quality assessment on the test script to be assessed based on the multi-dimensional quality indicators.

[0012] In a third aspect, an embodiment of the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a script quality assessment method as described in any one of the embodiments of the present invention is implemented.

[0013] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the script quality assessment method as described in any one of the embodiments of the present invention.

[0014] The embodiments of the present invention provide a script quality assessment method, device, electronic device and storage medium. The method tests the target software function containing positioning instrumentation by running the test script to be assessed, and outputs an instrumentation log through the positioning instrumentation. Then, the multi-dimensional quality indicators of the test script to be assessed are determined based on the instrumentation log. This method can facilitate the code location information and operation information of the target software function output to the instrumentation log by the positioning instrumentation, and accurately determine the quality indicators of the test script to be assessed from multiple dimensions. The embodiments of the present invention further perform quality assessment on the test script to be assessed based on the quality indicators of multiple dimensions. This method can comprehensively assess the quality of the corresponding test script without relying on manual labor, thereby ensuring the assessment accuracy of the test script and improving the assessment efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 This is a flow chart of a script quality assessment method provided by an embodiment of the present invention;

[0017] Figure 2 is another flowchart of the script quality assessment method provided by an embodiment of the present invention;

[0018] Figure 3 is another flowchart of the script quality assessment method provided by an embodiment of the present invention;

[0019] Figure 4 is another flowchart of the script quality assessment method provided by an embodiment of the present invention;

[0020] Figure 5 is another flowchart of the script quality assessment method provided by an embodiment of the present invention;

[0021] Figure 6 This is a structural intention of the script quality assessment device provided by an embodiment of the present invention;

[0022] Figure 7 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] Figure 1 This is a flow chart of a script quality assessment method provided by an embodiment of the present invention. This embodiment is applicable to scenarios where test scripts are assessed. This method can be performed by a script quality assessment device provided by an embodiment of the present invention, which can be implemented in software and / or hardware. In a specific embodiment, the device can be integrated into an electronic device, such as a computer, a server, etc. The following embodiments will be described using the device integrated into an electronic device as an example. Figure 1 , the method may specifically include the following steps:

[0026] Step 101: Test the target software function containing the positioning instrumentation by running the test script to be evaluated, and output the instrumentation log through the positioning instrumentation. This step can help accurately determine the quality indicators of the test script to be evaluated from multiple dimensions based on the instrumentation log.

[0027] Specifically, the test script to be evaluated may include one test script, or may include a batch of multiple test scripts.

[0028] Specifically, the target functional software may be newly coded and submitted by developers through the development management module.

[0029] Specifically, the positioning stub mentioned above can be understood as a stub code inserted by a developer when writing a code for developing a target software function, and used to mark the code line to which the program output log of the target software function belongs.

[0030] Specifically, the above-mentioned instrumentation log can be understood as a test running log including the code line to which the content of the positioning instrumentation mark belongs when the target software function containing the positioning instrumentation is tested by running the test script to be evaluated.

[0031] Optionally, the above process of testing the target software function containing positioning instrumentation by running the test script to be evaluated and outputting the instrumentation log through positioning instrumentation includes: testing the target software function containing positioning instrumentation by running the test script to be evaluated in multiple test environments respectively, and outputting the instrumentation log corresponding to each test environment through positioning instrumentation.

[0032] Optionally, the positioning instrumentation includes branch positioning instrumentation.

[0033] Optionally, the branch location instrumentation includes key branch location instrumentation.

[0034] Specifically, after completing the coding of the target software function, the developer may use a code parser to parse the original code, analyze program branches and key program branches, and add stub codes to each program branch and key program branch.

[0035] Specifically, program branches involving core business logic such as order status changes, permission verification, etc., or program branches containing multiple nested conditions or high complexity, such as branches involving logical judgments of multiple variables, can be identified as key program branches.

[0036] Optionally, the process of outputting the instrumentation log by positioning instrumentation includes: outputting the branch execution event to the instrumentation log by branch positioning instrumentation.

[0037] Specifically, the branch execution event of the corresponding branch can be output through the branch location instrumentation of each branch.

[0038] Optionally, the process of outputting branch execution events to the instrumentation log through branch location instrumentation includes: outputting key branch execution events to the instrumentation log through key branch location instrumentation.

[0039] Specifically, the corresponding key branch execution events can be output through the key branch positioning instrumentation of each key branch.

[0040] Optionally, the process of outputting the instrumentation log by positioning instrumentation includes: outputting the recovery status of the running environment data after the test is completed to the instrumentation log by positioning instrumentation.

[0041] Step 102: Determine the multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log. This step can accurately determine the quality indicators of the test script to be evaluated from multiple dimensions based on the code location information and operation information of the target software function output by the instrumentation in the instrumentation log.

[0042] Optionally, the multi-dimensional quality indicators include: coverage branch number score and assertion number score.

[0043] Specifically, the above-mentioned coverage branch number score can be understood as: an indicator used to characterize the number of branches of the target software function covered by the test when the target software function is tested by running the test script to be evaluated; the above-mentioned assertion number score can be understood as: a score used to characterize the number of assertions when the target software function is tested by running the test script to be evaluated.

[0044] Optionally, the multi-dimensional quality indicator includes a coverage score for key branches.

[0045] Specifically, the above-mentioned coverage key branch score can be understood as an indicator used to represent the key sub-index of the target software function covered by the test when the target software function is tested by running the test script to be evaluated.

[0046] Optionally, the multi-dimensional quality indicator includes a reusability score.

[0047] Specifically, the above reusability score can be understood as an indicator used to characterize the reusability of the test script to be evaluated in testing the target software function in different operating environments.

[0048] Optionally, the multi-dimensional quality indicator includes a data recovery score.

[0049] Specifically, the above data recovery score can be understood as an indicator used to represent the degree of recovery of the operating environment data after testing the target software function by running the test script to be evaluated.

[0050] Optionally, the multi-dimensional quality indicator includes a score for the number of problems found.

[0051] Specifically, the score of the number of problems found can be understood as an indicator used to represent the number of functional problems found by running the test script to be evaluated to test the target software function.

[0052] Optionally, the above process of determining the multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log includes: determining the proportion of executed branches in all branches of the target software function based on branch execution events to obtain the coverage branch ratio of the test script to be evaluated, and determining the coverage branch number score based on the coverage branch ratio; and determining the number of non-true assertions of the test script to be evaluated based on the instrumentation log, and determining the assertion number score based on the number of non-true assertions.

[0053] Optionally, the process of determining the proportion of executed branches in all branches of the target software function based on branch execution events to obtain the coverage branch ratio of the test script to be evaluated includes: determining the number of branches covered by the test script to be evaluated based on the branch execution events, and calculating the percentage of the number of covered branches in all branches of the target software function to obtain the above-mentioned coverage branch ratio.

[0054] Optionally, the process of determining the coverage branch number score based on the coverage branch ratio includes: normalizing the coverage branch ratio and determining the corresponding normalized value as the coverage branch number score or directly determining the coverage branch ratio as the coverage branch number score.

[0055] Optionally, the process of determining the number of non-always-true assertions of the test script to be evaluated based on the instrumentation log includes: determining the total number of assertions and the number of always-true assertions based on the instrumentation log, and calculating the difference between the total number of assertions and the number of always-true assertions to obtain the number of non-always-true assertions.

[0056] Optionally, the process of determining the assertion score based on the number of non-always true assertions includes: normalizing the number of non-always true assertions and determining the assertion score using the corresponding normalized value, or directly determining the assertion score using the number of non-always true assertions.

[0057] It is understandable that if the branch coverage is too low, the test script may have a large number of missed tests; if the number of non-always true assertions is small or there are a large number of always true assertions, it means that the test script has not effectively verified the key outputs and state changes of the program, and even if the test passes, there may actually be defects; therefore, determining the covered branch count score based on the branch coverage and determining the assertion count score based on the number of non-always true assertions can help to accurately evaluate the script quality of the test script based on the covered branch count score and the assertion count score.

[0058] Optionally, the above process of determining the multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log includes: determining the proportion of executed branches in all key branches of the target software function based on the key branch execution events to obtain the coverage key branch ratio; and determining the coverage key branch score based on the coverage key branch ratio.

[0059] Optionally, the number of critical branches covered by the test script to be evaluated is determined based on the critical branch execution events, and the percentage of the number of covered critical branches to all critical branches of the target software function is calculated to obtain the above-mentioned coverage critical branch ratio.

[0060] Optionally, the process of determining the coverage key branch score based on the coverage key branch ratio includes: after normalizing the coverage key branch ratio, determining the corresponding normalized value as the coverage key branch score, or determining the coverage key branch ratio as the coverage key branch score.

[0061] Understandably, if critical branches aren't covered, the test script may miss core program functionality or potential defects. For example, in an e-commerce system, missing the "payment successful" and "payment failed" branches could lead to undetected refund logic flaws. In a financial system, missing the "authorization verification passed" and "inadequate authorization" branches could lead to security vulnerabilities. Therefore, determining a critical branch coverage score based on the critical branch coverage ratio can help accurately evaluate the quality of test scripts based on the critical branch coverage score.

[0062] Optionally, the process of determining multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log includes: determining the test results corresponding to each test environment based on the corresponding instrumentation log; and determining the reusability score of the test script to be evaluated based on the consistency of each test result with other test results.

[0063] Optionally, the process of determining the multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log includes: determining a data recovery score based on the recovery status of the running environment data.

[0064] Step 103: Perform a quality assessment of the test script based on multi-dimensional quality indicators. This step, building on steps 101 and 102, comprehensively assesses the quality of the test script using accurate multi-dimensional quality indicators without relying on manual effort, thereby ensuring accuracy and improving efficiency in the test script assessment.

[0065] Optionally, the process of performing quality assessment on the test script to be assessed based on the multi-dimensional quality indicators includes: summing the values ​​of the multi-dimensional quality indicators and determining a quality assessment result of the test script to be assessed based on the corresponding sum values.

[0066] Specifically, the total quality score of the test script to be evaluated may be obtained by summing up the scores of the number of covered branches, the number of assertions, the score of key branches covered, the reusability score, the data recovery score, and the score of the number of problems found.

[0067] Specifically, the scores of the number of covered branches, the number of assertions, the score of key branches covered, the reusability score, the data recovery score and the score of the number of problems found can also be subjected to other common processing, such as standardization, and the standardized processing results can be summed to obtain the total quality score of the above-mentioned test script to be evaluated.

[0068] The following further introduces the script quality assessment method provided by the embodiment of the present invention. Figure 2 As shown, the following steps may be included:

[0069] Step 201 : testing target software functions including positioning instrumentation by running the test script to be evaluated in multiple test environments respectively, and outputting instrumentation logs corresponding to each test environment through positioning instrumentation.

[0070] Specifically, the above-mentioned multiple test environments may include test environments for different purposes, such as development environment, test environment, pre-production environment and production environment; they may also include environments with different technical architectures, such as stand-alone environment, distributed environment and containerized environment.

[0071] Optionally, the process of testing the target software function containing the positioning instrumentation by running the test script to be evaluated in multiple test environments, and outputting the instrumentation log corresponding to each test environment through the positioning instrumentation includes:

[0072] For each test environment, run the test script to be evaluated under different external state conditions of the current test environment to test the target software functions containing positioning instrumentation, and output the instrumentation logs corresponding to each external state condition of the current test environment through positioning instrumentation.

[0073] Step 202: Determine the test results corresponding to each test environment based on the corresponding instrumentation log.

[0074] Specifically, the above test results may include functional verification results, performance indicators, compatibility performance, data consistency and / or security test results, etc.

[0075] Specifically, the test results corresponding to the external condition states of the various test environments may also be determined based on the corresponding instrumentation logs.

[0076] Step 203: Determine the reusability score of the test script to be evaluated based on the consistency of each test result with other test results.

[0077] Optionally, the process of determining the reusability score of the test script to be evaluated based on the consistency of each test result with other test results includes:

[0078] The number of test results with consistent result values ​​is obtained, and the reusability score is determined based on the number of test results with consistent result values ​​and the total number of test results.

[0079] Specifically, the proportion of test results of the result values ​​in the test results may be calculated to obtain a consistent result ratio, and the reusable score may be determined based on the consistent result ratio.

[0080] Specifically, the proportion of consistent results may be directly determined as the reusability score, or the proportion of consistent results may be normalized to obtain the reusability score.

[0081] Optionally, when the test script to be evaluated includes a batch of multiple test scripts, the process of determining the reusability score of the test script to be evaluated based on the consistency of each test result with other test results includes:

[0082] For each test script, when the corresponding test results are consistent, the current test script is determined to be a reusable test script; and the proportion of reusable test scripts in a batch of multiple test scripts determines the above reusability score.

[0083] Specifically, the proportion of reusable test scripts in the batch of multiple test scripts may be directly determined as the reusability score, or the proportion of reusable test scripts in the batch of multiple test scripts may be standardized to obtain the reusability score.

[0084] Step 204 : Perform quality assessment on the test script to be assessed based on the reusability score.

[0085] It can be understood that when the test results of a test case on the target software function in multiple test environments are consistent, it indicates that the test case has good environmental compatibility. Therefore, determining the reusability score based on the test results of the test case running in each test environment and evaluating the test script based on the reusability score can improve the accuracy of the quality assessment of the test script.

[0086] The following further introduces the script quality assessment method provided by the embodiment of the present invention. Figure 3 As shown, the following steps may be included:

[0087] Step 301 : testing the target software function including the positioning instrumentation by running the test script to be evaluated, and outputting the recovery status of the running environment data after the test is completed to the instrumentation log through the positioning instrumentation.

[0088] It is understood that after testing the target software functionality, to prevent data generated during the test from interfering with subsequent normal operations (e.g., data contamination, configuration conflicts, etc.), it is necessary to restore the system environment and data state to the initial or standard state before the test, including all data and environmental parameters involved. This operational environment data can be understood as all types of data and configuration information that need to be restored or reset.

[0089] Step 302: Determine a data recovery score based on the recovery status of the operating environment data.

[0090] Specifically, the recovery status of the above operating environment data can be restored or not restored.

[0091] Optionally, the process of determining the data recovery score based on the recovery status of the operating environment data includes: determining the data recovery score based on a ratio of the recovered operating environment data to the total operating environment data.

[0092] Specifically, the proportion of the restored operating environment data station to the total operating environment data may be directly determined as the data recovery score, or the proportion of the restored operating environment data station to the total operating environment data may be standardized to obtain the data recovery score.

[0093] Step 303: Determine the quality of the test script to be evaluated based on the data recovery score.

[0094] It is understandable that if the operating environment data is not restored after functional testing, it may have a negative impact on many aspects such as system stability, business logic, and data accuracy. Therefore, determining the data recovery score based on the recovery status of the operating environment data and conducting quality assessment of the test script based on the data recovery score can improve the accuracy of the quality assessment.

[0095] The following further introduces the script quality assessment method provided by the embodiment of the present invention. Figure 4 As shown, Figure 1 Step 102 in the embodiment may include the following steps:

[0096] Step 1021 : Determine functional issues of the target software function based on the instrumentation log.

[0097] Specifically, the problems with the above-mentioned target software functions may include: function loss, function redundancy, function implementation deviation, function logic error, data processing anomaly, function flow error, data security vulnerability and / or state transition error.

[0098] Step 1022: Determine the severity of the functional problem, and determine a score for the number of discovered problems based on the number of functional problems of different severities and the problem score weights.

[0099] Specifically, the severity of the above functional problems includes: fatal, serious, general and recommended.

[0100] Specifically, functional issues with a critical severity level include those that may cause system crashes or paralysis of core functions, such as core function failures or data security vulnerabilities.

[0101] Specifically, functional issues with a serious severity level include those that affect core business but the system can still operate, such as functional logic errors or data processing anomalies.

[0102] Specifically, issues with a general severity level include those that affect the user experience but do not block the process, such as abnormal but usable functions.

[0103] Specifically, issues with a severity level of "suggestion" include those that do not affect functionality but can optimize the experience, such as cumbersome operating procedures or unfriendly prompts.

[0104] Specifically, the score weights of the above questions may be pre-set, and the score weights of the four severity levels of fatal, serious, general and recommended may be set in descending order.

[0105] Specifically, the weights of general and recommended may be set to be the same.

[0106] Optionally, the process of determining the score of the number of found problems based on the number of functional problems of different severities and the problem score weights includes: performing weighted summation of the number of functional problems of different severities based on the corresponding problem score weights to obtain the score of the number of found problems.

[0107] Specifically, the score weights of various severity levels may be adjusted based on the total number of fatal problems and serious problems, and the number of functional problems of different severity levels may be weighted and summed based on the adjusted weights to obtain the above-mentioned score of the number of problems found.

[0108] It is understandable that when the target software function does have corresponding problems, if the test script usually finds a large number of valid and serious problems (such as fatal / critical functional errors), it means that the script covers the core business scenarios and has a high quality; on the contrary, if the number of problems is large but mainly "suggestion level" or "repeated problems" (such as interface style confusion), it may reflect that the script design focuses on non-core details and the quality needs to be re-evaluated. Therefore, the embodiment of the present invention determines the score of the number of found problems based on the number and score weight of functional problems of different severities, which can help to accurately evaluate the quality of the test script based on the score of the found problem book.

[0109] The script quality assessment method provided by the embodiment of the present invention is further described below. Figure 5 As shown, the following steps may be included:

[0110] Step 501 : testing the target software function including the positioning instrumentation by running the test script to be evaluated, and outputting the instrumentation log through the positioning instrumentation.

[0111] Step 502: Determine multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log.

[0112] Step 503: Determine the evaluation weight of each dimension quality indicator based on the functional characteristics of the target software function.

[0113] Optionally, the functional characteristics of the target software include improvement goals of software functions, business logic complexity, state transition frequency, and / or similar functional test script characteristics.

[0114] Specifically, the above-mentioned similar function testing features can be understood as the features detected by high-quality test scripts that test similar functions of the target software functions. For example, when the number of non-true assertions in the corresponding high-quality script is more than that in the conventional test script, the evaluation weight of the assertion number score is increased.

[0115] Specifically, the number of assertions in the conventional test script may be determined based on empirical data, or may be determined based on multiple test scripts of different types of functions.

[0116] Optionally, the above process of determining the evaluation weights of the quality indicators of each dimension based on the functional characteristics of the target software function includes: querying a pre-established feature weight mapping table through the functional characteristics of the target software function, or calculating the evaluation weights of the above-mentioned quality indicators of each dimension through a pre-established weight function based on the functional characteristics of the target software function.

[0117] Step 504 , performing weighted summation on the values ​​of the quality indicators of each dimension based on the corresponding evaluation weights to obtain a total quality score of the test script to be evaluated, and determining a quality evaluation result of the test script to be evaluated based on the total quality score.

[0118] It is understandable that for test scripts used to test software functions with different functional characteristics, the quality indicator requirements for each dimension are different. Therefore, based on the functional characteristics of software functions with different characteristics, the evaluation weights of the quality indicators of each dimension are determined. Then, based on the corresponding evaluation weights, the quality indicators of each dimension are weighted and summed to obtain the total quality score. The quality evaluation results of the test script to be evaluated are determined based on the total quality score, so that the quality of software functions with different functional characteristics can be accurately evaluated.

[0119] The embodiment of the present invention can dynamically adjust the weights of quality indicators of various dimensions according to the target software function, can accurately evaluate the quality of the target software function, and can facilitate effective improvement of the target software function.

[0120] In an optional specific example, the script quality evaluation device provided by an embodiment of the present invention also includes generating a test script evaluation report based on the value of the quality indicator of each dimension and the total quality score, and pushing the test script evaluation report to the target person.

[0121] Figure 6 This is a structural diagram of a script quality assessment device provided by an embodiment of the present invention, which is suitable for executing the script quality assessment method provided by an embodiment of the present invention. Figure 6 As shown, the device may specifically include:

[0122] The test execution and log acquisition module 601 is used to test the target software functions containing positioning instrumentation by running the test script to be evaluated, and output the instrumentation log through positioning instrumentation. This module can help accurately determine the quality indicators of the test script to be evaluated from multiple dimensions based on the instrumentation log.

[0123] Optionally, the positioning instrumentation includes branch positioning instrumentation.

[0124] Optionally, the branch location instrumentation includes key branch location instrumentation.

[0125] Optionally, the test execution and log acquisition module 601 can be specifically configured to output branch execution events to the instrumentation log through branch location instrumentation.

[0126] Optionally, the test execution and log acquisition module 601 can be specifically configured to output key branch execution events to the instrumentation log through key branch location instrumentation.

[0127] Optionally, the test running and log acquisition module 601 can be specifically used to test the target software function containing positioning instrumentation by running the test script to be evaluated in multiple test environments, and output the instrumentation log corresponding to each test environment through positioning instrumentation.

[0128] Optionally, the test execution and log acquisition module 601 can be specifically configured to output the recovery status of the execution environment data after the test is completed to the instrumentation log by positioning the instrumentation.

[0129] The quality indicator acquisition module 602 is used to determine the multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log. This module can accurately determine the quality indicators of the test script to be evaluated from multiple dimensions based on the code location information and operation information of the target software function output by the instrumentation in the instrumentation log.

[0130] Optionally, the multi-dimensional quality indicators include: coverage branch number score and assertion number score.

[0131] Optionally, the multi-dimensional quality indicator includes a coverage score for key branches.

[0132] Optionally, the multi-dimensional quality indicator includes a reusability score.

[0133] Optionally, the multi-dimensional quality indicator includes a data recovery score.

[0134] Optionally, the multi-dimensional quality indicator includes a score for the number of problems found.

[0135] Optionally, the above-mentioned quality indicator acquisition module 602 can be specifically used to determine the proportion of executed branches in all branches of the target software function based on branch execution events to obtain the coverage branch ratio of the test script to be evaluated, and determine the coverage branch number score based on the coverage branch ratio; and determine the number of non-true assertions of the test script to be evaluated based on the instrumentation log, and determine the assertion number score based on the number of non-true assertions.

[0136] Optionally, the quality indicator acquisition module 602 can be specifically used to determine the proportion of executed branches in all key branches of the target software function based on key branch execution events to obtain a coverage key branch ratio; and determine a coverage key branch score based on the coverage key branch ratio.

[0137] Optionally, the quality indicator acquisition module 602 can be specifically used to determine the test results corresponding to each test environment based on the corresponding instrumentation log; and determine the reusability score of the test script to be evaluated based on the consistency of each test result with other test results.

[0138] Optionally, the quality indicator acquisition module 602 can be specifically configured to determine a data recovery score based on the recovery status of the operating environment data.

[0139] Optionally, the quality indicator acquisition module 602 can be specifically used to determine functional problems of the target software function based on the instrumentation log; and determine the severity of the functional problems, and determine the score of the number of found problems based on the number of functional problems of different severities and the problem score weights.

[0140] The quality assessment module 603 is used to perform quality assessment on the test script to be assessed based on multi-dimensional quality indicators. This module can be combined with modules 601 and 602 to test the target software function containing the positioning instrumentation by running the test script to be assessed, and output the instrumentation log through the positioning instrumentation. Then, the multi-dimensional quality indicators of the test script to be assessed are determined based on the instrumentation log. The quality indicators of the test script to be assessed can be accurately determined from multiple dimensions based on the code location information and operation information of the target software function output by the positioning instrumentation to the instrumentation log. The embodiment of the present invention further performs quality assessment on the test script to be assessed based on the quality indicators of multiple dimensions, and can comprehensively assess the quality of the corresponding test script without relying on manual labor, thereby ensuring the accuracy of the assessment of the test script and improving the efficiency of the assessment.

[0141] Optionally, the above-mentioned quality assessment module 603 can be specifically used to determine the assessment weights of the quality indicators of each dimension based on the functional characteristics of the target software function; and to perform weighted summation on the values ​​of the quality indicators of each dimension based on the corresponding assessment weights to obtain the total quality score of the test script to be evaluated, and determine the quality assessment result of the test script to be evaluated based on the total quality score.

[0142] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0143] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the script quality assessment method provided in any of the above embodiments when executing the program.

[0144] An embodiment of the present invention further provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the script quality assessment method provided in any of the above embodiments.

[0145] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the script quality assessment method as described in any one of the embodiments of the present invention.

[0146] Reference below Figure 7 , which shows a schematic structural diagram of a computer system 700 of an electronic device suitable for implementing an embodiment of the present invention. Figure 7 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0147] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the system 700 are also stored in the RAM 703. The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0148] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output section 707 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 708 including a hard disk; and a communication section 709 including a network interface card such as a LAN card or a modem. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 710 as needed, so that computer programs read therefrom can be installed into the storage section 708 as needed.

[0149] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from a removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0150] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0151] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0152] The modules and / or units described in the embodiments of the present invention may be implemented in software or hardware. The modules and / or units described may also be provided in a processor. For example, a processor may be described as including a test execution and log acquisition module, a quality indicator acquisition module, and a quality assessment module. The names of these modules do not, in some cases, limit the modules themselves.

[0153] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment, or may exist independently without being incorporated into the device. The computer-readable medium carries one or more programs, and when executed by the device, the device performs the following steps: testing a target software function containing location instrumentation by running a test script to be evaluated, and outputting an instrumentation log using the location instrumentation; determining multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log; and performing a quality assessment on the test script to be evaluated based on the multi-dimensional quality indicators.

[0154] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A script quality assessment method, characterized in that: include: Test the target software function containing the positioning instrumentation by running the test script to be evaluated, and output the instrumentation log through the positioning instrumentation; Determining multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log; and Performing a quality assessment on the test script to be assessed based on the multi-dimensional quality indicators.

2. The script quality assessment method according to claim 1, wherein: The multi-dimensional quality indicators include: coverage branch number score and assertion number score; the positioning instrumentation includes branch positioning instrumentation; Outputting the instrumentation log through the positioning instrumentation includes: Outputting branch execution events to the instrumentation log through the branch location instrumentation; Determining the multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log includes: Determining the proportion of executed branches in all branches of the target software function based on the branch execution events to obtain a coverage branch ratio of the test script to be evaluated, and determining the coverage branch number score based on the coverage branch ratio; and The number of non-true assertions of the test script to be evaluated is determined based on the instrumentation log, and the assertion score is determined based on the number of non-true assertions.

3. The script quality assessment method according to claim 2, characterized in that: The branch location instrumentation includes key branch location instrumentation, and the multi-dimensional quality indicator includes a coverage key branch score; Outputting the branch execution event to the instrumentation log through the branch location instrumentation includes: Outputting key branch execution events to the instrumentation log through the key branch positioning instrumentation; Determining the multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log includes: Determining the proportion of executed branches in all key branches of the target software function based on the key branch execution event to obtain a coverage key branch ratio; and The coverage critical branch score is determined based on the coverage critical branch ratio.

4. The script quality assessment method according to any one of claims 1 to 3, characterized in that: The multi-dimensional quality indicator includes a reusability score. The method of testing the target software function including the positioning instrumentation by running the test script to be evaluated and outputting the instrumentation log through the positioning instrumentation includes: Testing target software functions containing positioning instrumentation by running the test script to be evaluated in multiple test environments respectively, and outputting instrumentation logs corresponding to each test environment through the positioning instrumentation; Determining the multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log includes: Determine the test results for each test environment based on the corresponding instrumentation logs; and Based on the consistency of each test result with other test results, the reusability score of the test script to be evaluated is determined.

5. The script quality assessment method according to any one of claims 1 to 3, characterized in that: The multi-dimensional quality indicators include a data recovery score; Outputting the instrumentation log through the positioning instrumentation includes: Outputting the recovery status of the running environment data after the test is completed to the instrumentation log through the positioning instrumentation; Determining the multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log includes: The data recovery score is determined based on the recovery status of the operating environment data.

6. The script quality assessment method according to any one of claims 1 to 3, characterized in that: The multi-dimensional quality indicators include a score for the number of problems found; Determining the multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log includes: Determining a functional problem of a target software function based on the instrumentation log; as well as The severity of the functional problem is determined, and the score of the number of found problems is determined based on the number of functional problems of different severities and the problem score weights.

7. The script quality assessment method according to claim 1, wherein: The performing quality assessment on the test script to be assessed based on the multi-dimensional quality indicators includes: Determining the evaluation weights of the quality indicators of each dimension based on the functional characteristics of the target software function; and The values ​​of the quality indicators of each dimension are weighted and summed based on the corresponding evaluation weights to obtain the total quality score of the test script to be evaluated, and the quality evaluation result of the test script to be evaluated is determined based on the total quality score.

8. A script quality assessment device, characterized in that: include: A test running and log acquisition module, configured to test target software functions containing positioning instrumentation by running a test script to be evaluated, and output instrumentation logs through the positioning instrumentation; A quality indicator acquisition module, configured to determine multi-dimensional quality indicators of the test script to be evaluated based on the instrumentation log; as well as A quality assessment module is used to perform quality assessment on the test script to be assessed based on the multi-dimensional quality indicators.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the script quality assessment method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the script quality assessment method according to any one of claims 1 to 7 is implemented.

Images