Software project automation test efficiency analysis method and apparatus, and electronic device

Abstract

The invention discloses a software project automation test efficiency analysis method and device and electronic equipment, and relates to the financial science and technology field or other related technology fields, the method comprises the steps that a pre-constructed index system model is called, the index system model at least comprises a test input index, an intermediate product and a test output index, based on the index system model, using a network efficiency DEA model to evaluate the static efficiency, and analyzing automatic test efficiency values of each development project team related to the target software project in a plurality of preset historical time periods in a predetermined historical period; and based on the index system model, analyzing the dynamic change of the automatic test efficiency value by using a dynamic Malmquist index, and outputting a test efficiency evaluation report. According to the method and the device, the technical problem of low evaluation accuracy due to dependence on qualitative or semi-quantitative indexes during software project automatic test efficiency evaluation in the related technology is solved.

Description

Technical Field

[0001] This invention relates to the field of financial technology or other related fields, and more specifically, to a method and apparatus for analyzing the efficiency of automated testing of software projects, and an electronic device. Background Technology

[0002] With the rapid development of information technology and software development, automated testing has become a key means to improve software quality and accelerate development cycles. Compared to traditional manual testing, automated testing can significantly improve testing efficiency and coverage, especially in regression testing and continuous integration. However, implementing automated testing is not without cost; it involves initial investment in framework construction, ongoing script development and maintenance costs, and the need for advanced skills among testers. Therefore, accurately measuring the efficiency of automated testing is crucial for the rational allocation of resources and optimization of testing strategies.

[0003] In related technologies, the evaluation of automated testing efficiency mostly relies on qualitative or semi-quantitative indicators, such as the input-output ratio of automated testing. This simple ratio calculation often ignores the complexity of the testing process and the multi-dimensional influencing factors. It is difficult to capture the trend of automated testing efficiency evolving over time, as well as the contribution of technological innovation to efficiency improvement, resulting in evaluation results that lack depth and foresight.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method, apparatus, and electronic device for analyzing the efficiency of automated testing in software projects, to at least solve the technical problem of low accuracy in evaluating the efficiency of automated testing in software projects, which relies on qualitative or semi-quantitative indicators.

[0006] To achieve the above objectives, according to one aspect of this application, a method for analyzing the efficiency of automated testing in software projects is provided, comprising: calling a pre-built indicator system model, wherein the indicator system model includes at least: test input indicators, intermediate products, and test output indicators, wherein the test input indicators include: environment setup costs and the number of testers, the intermediate products include: the number of software test cases, and the test output indicators include: the number of problems found and the average test execution time; based on the indicator system model, using a network efficiency DEA model to evaluate static efficiency, and analyzing the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period; based on the indicator system model, using a dynamic Malmquist index to analyze the dynamic changes of the automated testing efficiency values, and outputting a test efficiency evaluation report, wherein the test efficiency report includes at least: the automated testing efficiency values ​​of the development project teams in each historical time period, and the dynamic efficiency change trend during the automated testing implementation process in the predetermined historical time period.

[0007] Optionally, the steps of evaluating static efficiency using a network efficiency DEA model include: decomposing the automated testing process of a software project into a first stage and a second stage, wherein the first stage is the stage of setting up the software project environment and configuring resources, and the second stage is the stage of developing and executing test cases; using the network efficiency DEA model to generate the number of software test cases in the first stage based on the environment setup cost and the number of testers, and generating the number of problems found and the average test execution time in the second stage based on the number of software test cases; introducing predetermined dummy variables, using the network efficiency DEA model to use the predetermined dummy variables as reconciliation variables between the first stage and the second stage, and calculating the efficiency value of the first stage and the efficiency value of the second stage; based on the efficiency value of the first stage and the weight parameters corresponding to the first stage, and the efficiency value of the second stage and the weight parameters corresponding to the second stage, analyzing the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period.

[0008] Optionally, when using the network efficiency DEA model, the method further includes: for each development project team, ranking the efficiency values ​​of the multiple automated test teams using the network efficiency DEA model, wherein the ranking method includes: overall efficiency value ranking, first-stage efficiency value ranking, and second-stage efficiency value ranking, wherein the overall efficiency value includes the first-stage efficiency value and the second-stage efficiency value; and based on the efficiency value ranking result, determining the target stage in which the efficiency value decreases, wherein the target stage is either the first stage or the second stage.

[0009] Optionally, the step of analyzing the dynamic changes in the automated testing efficiency value using the dynamic Malmquist index and outputting a test efficiency evaluation report includes: acquiring test panel data containing time series; analyzing the test panel data using the dynamic Malmquist index to obtain software project development technology efficiency parameters and efficiency fluctuation parameters containing time series during the automated testing process; and outputting the test efficiency evaluation report based on the software project development technology efficiency parameters and efficiency fluctuation parameters in the time series.

[0010] Optionally, the step of analyzing the test panel data using the dynamic Malmquist index to obtain time-series software project development technology efficiency parameters and efficiency fluctuation parameters during the automated testing process includes: analyzing the test panel data using the dynamic Malmquist index to obtain a dynamic index set; decomposing the dynamic index set into time-series software project development technology efficiency parameters and software project development technology progress parameters; and determining the efficiency fluctuation parameters of the automated testing process at different historical time points based on the decomposed software project development technology efficiency parameters and software project development technology progress parameters.

[0011] Optionally, the step of outputting the test efficiency evaluation report based on the software project development technical efficiency parameters and efficiency fluctuation parameters of the time series includes: retrieving the automated testing efficiency values ​​of each development project team involved in the target software project in different preset historical time periods after evaluating static efficiency using the network efficiency DEA model; retrieving the efficiency fluctuation parameters of the automated testing process at different historical time points using the dynamic Malmquist index analysis; unifying the time axis of evaluating static efficiency using the network efficiency DEA model and analyzing the dynamic changes of the automated testing efficiency values ​​using the dynamic Malmquist index analysis, to obtain the test efficiency evaluation report containing the automated testing efficiency values ​​and dynamic efficiency change trends of each historical time period.

[0012] Optionally, after outputting the test efficiency evaluation report based on the software project development technical efficiency parameters and efficiency fluctuation parameters of the time series, the method further includes: analyzing the target teams in each development project team that have experienced efficiency decline based on the automated test efficiency values ​​and dynamic efficiency change trends of each historical time period in the test efficiency evaluation report; querying the contact information corresponding to the team leader positions of the target teams; and outputting automated test efficiency decline prompts and automated test optimization suggestions to the contact information.

[0013] According to another aspect of the present invention, an analysis device for the efficiency of automated testing of software projects is also provided, comprising: an indicator system model invocation unit, configured to invoke a pre-built indicator system model, wherein the indicator system model includes at least: test input indicators, intermediate products, and test output indicators, wherein the test input indicators include: environment setup costs and the number of testers, the intermediate products include: the number of software test cases, and the test output indicators include: the number of problems found and the average test execution time; a static test efficiency evaluation unit, configured to evaluate static efficiency based on the indicator system model using a network efficiency DEA model, and analyze the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period; and a dynamic test change analysis unit, configured to analyze the dynamic changes of the automated testing efficiency values ​​based on the indicator system model using a dynamic Malmquist index, and output a test efficiency evaluation report, wherein the test efficiency report includes at least: the automated testing efficiency values ​​of the development project teams in each historical time period, and the dynamic efficiency change trend during the automated testing implementation process in the predetermined historical time period.

[0014] Optionally, the static efficiency evaluation unit includes: a first decomposition module, used to decompose the automated testing process of the software project into a first stage and a second stage, wherein the first stage is the stage of setting up the software project environment and configuring resources, and the second stage is the stage of developing and executing test cases; a phased generation module, used to generate the number of software test cases in the first stage based on the environment setup cost and the number of testers, and to generate the number of problems found and the average test execution time in the second stage based on the number of software test cases; a dummy variable usage module, used to introduce predetermined dummy variables, use the network efficiency DEA model to use the predetermined dummy variables as coordination variables between the first stage and the second stage, and calculate the efficiency value of the first stage and the efficiency value of the second stage; and a first analysis module, used to analyze the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period, based on the efficiency value of the first stage and the weight parameters corresponding to the first stage, and the efficiency value of the second stage and the weight parameters corresponding to the second stage.

[0015] Optionally, when using the network efficiency DEA model, the static efficiency evaluation unit further includes: a first sorting module, used to sort the efficiency values ​​of the multiple development project teams for each development project team using the multiple automated test efficiency values ​​obtained from the network efficiency DEA model analysis, wherein the sorting method includes: overall efficiency value sorting, first-stage efficiency value sorting, and second-stage efficiency value sorting, the overall efficiency value including the first-stage efficiency value and the second-stage efficiency value; and an efficiency decline analysis module, used to determine the target stage in which the efficiency value decline occurs based on the efficiency value sorting results, wherein the target stage is the first stage or the second stage.

[0016] Optionally, the test dynamic change analysis unit includes: a test panel data acquisition module for acquiring test panel data containing time series; a data analysis module for analyzing the test panel data using a dynamic Malmquist index to obtain software project development technology efficiency parameters and efficiency fluctuation parameters containing time series during automated testing; and a report output module for outputting the test efficiency evaluation report based on the software project development technology efficiency parameters and efficiency fluctuation parameters of the time series.

[0017] Optionally, the data analysis module includes: a data analysis submodule, used to analyze the test panel data using the dynamic Malmquist index to obtain a dynamic index set; a second decomposition module, used to decompose the dynamic index set into software project development technology efficiency parameters and software project development technology progress parameters containing time series; and a parameter determination submodule, used to determine the efficiency fluctuation parameters of the automated testing process at different historical time points based on the decomposed software project development technology efficiency parameters and software project development technology progress parameters.

[0018] Optionally, the report output module includes: a first retrieval submodule, used to retrieve the automated testing efficiency values ​​of each development project team involved in the target software project in different preset historical time periods after evaluating static efficiency using the network efficiency DEA model; a second retrieval submodule, used to retrieve the efficiency fluctuation parameters of the automated testing process at different historical time points using the dynamic Malmquist index analysis; and a time axis unification submodule, used to unify the time axis of the static efficiency evaluation using the network efficiency DEA model and the dynamic changes of the automated testing efficiency values ​​using the dynamic Malmquist index analysis, to obtain the test efficiency evaluation report containing the automated testing efficiency values ​​and dynamic efficiency change trends for each historical time period.

[0019] Optionally, the software project automated testing efficiency analysis device further includes: a team analysis unit, used to analyze the target teams in each development project team that have experienced efficiency decline based on the automated testing efficiency values ​​and dynamic efficiency change trends of each historical time period in the test efficiency evaluation report after outputting the test efficiency evaluation report according to the software project development technical efficiency parameters and efficiency fluctuation parameters of the time series; and an optimization suggestion unit, used to query the contact information corresponding to the team responsible positions of the target teams, and output automated testing efficiency decline prompts and automated testing optimization suggestions to the contact information.

[0020] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, the device where the computer-readable storage medium is located executes the method for analyzing the efficiency of automated testing of software projects as described above.

[0021] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors enable the one or more processors to implement the software project automated testing efficiency analysis method described above.

[0022] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the method for analyzing the efficiency of automated testing of software projects as described in any one of the above embodiments.

[0023] In this disclosure, a pre-built indicator system model can be invoked. This model includes at least: test input indicators, intermediate products, and test output indicators. Test input indicators include: environment setup costs and the number of testers. Intermediate products include: the number of software test cases. Test output indicators include: the number of issues found and the average test execution time. Based on the indicator system model, a network efficiency DEA model is used to evaluate static efficiency, analyzing the automated testing efficiency values ​​of each development project team involved in the target software project across multiple preset historical time periods within a predetermined historical period. Based on the indicator system model, a dynamic Malmquist index is used to analyze the dynamic changes in automated testing efficiency values, outputting a test efficiency evaluation report. This report includes at least: the automated testing efficiency values ​​of the development project teams in each historical time period, and the dynamic efficiency change trend during the automated testing implementation process within the predetermined historical time period.

[0024] Based on the above-disclosed content, a comprehensive test efficiency evaluation report can be generated through static efficiency assessment of network DEA and time series analysis of the dynamic Malmquist index. By utilizing the dynamic Malmquist index, this invention can monitor the long-term trend of automated testing efficiency, comprehensively evaluate automated testing efficiency from both static and dynamic dimensions, provide multi-angle efficiency analysis results, and greatly improve the accuracy of the evaluation. This solves the technical problem of low accuracy in evaluating the efficiency of automated testing of software projects, which relies on qualitative or semi-quantitative indicators. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0026] Figure 1 A hardware block diagram of a computer terminal (or mobile device) for an analysis method to improve the efficiency of automated testing in software projects is shown.

[0027] Figure 2 This is a flowchart of an optional method for analyzing the efficiency of automated testing of software projects according to an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of an optional software project automated testing efficiency analysis device according to an embodiment of the present invention;

[0029] Figure 4 This is a structural block diagram of an electronic device for analyzing the efficiency of automated testing of software projects according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] To facilitate understanding of the present invention by those skilled in the art, some terms or nouns involved in the various embodiments of the present invention are explained below:

[0033] Network Data Envelopment Analysis (NDEA) is a data analysis method used to evaluate the efficiency of decision-making units comprising multiple stages. Compared to traditional DEA methods, NDEA can analyze the efficiency of different stages within a decision-making unit and their interactions in greater detail. In this invention, by decomposing the automated testing process into two consecutive stages, Network DEA can evaluate the efficiency of each stage separately, as well as the efficiency of connections between stages, thereby providing more comprehensive efficiency analysis results.

[0034] The Dynamic Malmquist Index (DMI) is used for panel data analysis. It can be decomposed into an index of change in technical efficiency and an index of technological progress to assess the efficiency changes of a decision-making unit (such as a testing team) at different points in time.

[0035] A Decision Making Unit (DMU) is an object evaluated in efficiency and productivity analysis; it can be a department or a project team. In this invention, the DMU refers to the different development project teams participating in automated testing of a software project, serving as the basic unit for efficiency analysis.

[0036] It should be noted that the method and apparatus for analyzing the efficiency of automated testing of software projects disclosed herein can be used in the field of fintech for analyzing the efficiency of automated testing of software development projects based on fintech, and can also be used in any field other than fintech for analyzing the efficiency of automated testing of software development projects based on fintech. This disclosure does not limit the application field of the method and apparatus for analyzing the efficiency of automated testing of software projects.

[0037] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) collected in this public disclosure are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0038] It should be noted that in this disclosure, customer information is collected and analyzed, and users are provided with corresponding operation entry points to choose whether to agree to or reject the automated decision results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0039] The following embodiments of the present invention can be applied to systems / applications / devices for analyzing the efficiency of automated testing in various software projects. The present invention is applicable to software engineering scenarios, especially to the analysis of automated testing efficiency during software development and testing. For example, in large-scale software development projects, it can evaluate the efficiency of automated testing by different teams in large projects involving collaborative work among multiple development teams, providing a basis for optimal resource allocation.

[0040] This invention combines network DEA and the dynamic Malmquist index to comprehensively evaluate automated testing efficiency from both static and dynamic dimensions, providing multi-faceted efficiency analysis results. Through stage efficiency decomposition, this invention can accurately pinpoint automated testing efficiency bottlenecks, clarifying whether inefficiency is caused by environment setup, test case design, execution, or maintenance. Utilizing the dynamic Malmquist index, this invention can monitor long-term trends in automated testing efficiency, identifying specific periods of efficiency improvement or decline, as well as the underlying driving factors.

[0041] The present invention will now be described in detail with reference to various embodiments.

[0042] Example 1

[0043] According to an embodiment of the present invention, an embodiment of a method for analyzing the efficiency of automated testing of software projects is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0044] The method for analyzing the efficiency of automated testing of software projects provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for an analytical method to improve the efficiency of automated testing in software projects is shown. Figure 1 As shown, computer terminal 10 (or mobile device) may include one or more ( Figure 1 (Illustrated as 102a, 102b, ..., 102n) Processor 102 (processor 102 may include, but is not limited to, a microprocessor MCU (Microcontroller Unit) or a programmable gate array (FPGA), etc.), memory 104 for storing data, and transmission device 106 for communication functions. In addition, it may include: a display, input / output interface (I / O interface), Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), network interface, power supply, and / or camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0045] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0046] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the software project automated testing efficiency analysis method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned software project automated testing efficiency analysis method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0047] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0048] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0049] Under the aforementioned operating environment, this application provides the following: Figure 2 The method for analyzing the efficiency of automated testing in software projects is shown. Figure 2 This is a flowchart of an optional method for analyzing the efficiency of automated testing in software projects according to an embodiment of the present invention, such as... Figure 2 As shown, the method includes the following steps:

[0050] Step S201: Invoke the pre-built indicator system model, which includes at least: test input indicators, intermediate products and test output indicators. Test input indicators include: environment setup cost and number of testers. Intermediate products include: number of software test cases. Test output indicators include: number of problems found and average test execution time.

[0051] In this embodiment, when starting the automated testing efficiency evaluation process, a pre-built indicator system model is first invoked. This model is based on a deep understanding and analysis of the automated testing process and is designed as a quantitative tool to comprehensively measure the efficiency of automated testing, ensuring the accuracy and reliability of the evaluation.

[0052] Among them, the testing input indicator is the testing input indicator part of the indicator system model, which can provide the resource consumption in the automated testing process. Specifically, the environment setup cost covers the one-time costs required to build the automated testing environment, such as hardware equipment purchase, software licensing fees, and technical consulting fees. The number of testers reflects the number of engineers participating in automated testing, which is directly related to labor costs and test execution efficiency.

[0053] In this embodiment, the intermediate product specifically refers to the number of software test cases, which represents the breadth of coverage of automated testing, i.e., the number of test scenarios that the automated test scripts can execute. It should be noted that the number of test cases in this embodiment is a key indicator connecting testing input and output, reflecting the sufficiency and effectiveness of the automated test design.

[0054] Furthermore, test output metrics are the specific manifestations of automated testing results, including the number of issues found and the average test execution time. It's important to note that the number of issues found refers to the total number of software defects or problems detected during automated testing, a key indicator for measuring test quality. The average test execution time, on the other hand, reflects the efficiency of the testing process, i.e., the time required to complete a full suite of automated tests. These two output metrics are directly related to the economic and time efficiency of testing, helping to evaluate the overall efficiency of automated testing in the software project development process.

[0055] It should be noted that in the process of constructing the indicator system model in this embodiment, objective and quantifiable indicators can be selected to ensure that the evaluation results are scientific and effective. The indicator system comprehensively considers all stages of automated testing, including the initial environment setup, test case design, test execution, and problem detection, to ensure the comprehensiveness and systematicness of the evaluation.

[0056] Step S202: Based on the indicator system model, use the network efficiency DEA model to evaluate static efficiency and analyze the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period.

[0057] In step S202 of this embodiment, the core activity is to use Network Data Envelopment Analysis (NDEA) to evaluate the static efficiency of automated testing based on the indicator system model called in the first step. That is, through mathematical modeling, the efficiency performance of each development project team in automated testing at different time points within a predetermined historical period is objectively quantified to reveal the efficiency status under static time slices.

[0058] Optionally, the steps for evaluating static efficiency using a network efficiency DEA model include: decomposing the automated testing process of a software project into a first phase and a second phase, wherein the first phase is the software project environment setup and resource configuration phase, and the second phase is the test case development and execution phase; using the network efficiency DEA model to generate the number of software test cases in the first phase based on the environment setup cost and the number of testers, and generating the number of issues found and the average test execution time in the second phase based on the number of software test cases; introducing predetermined dummy variables, using the network efficiency DEA model to use the predetermined dummy variables as reconciliation variables between the first and second phases, and calculating the efficiency values ​​of the first and second phases; based on the efficiency value of the first phase and the corresponding weight parameters of the first phase, and the efficiency value of the second phase and the corresponding weight parameters of the second phase, analyzing the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period.

[0059] In the first phase, corresponding to the software project environment setup and resource allocation stage, the inputs are mainly environment setup costs and the number of testers, while the output is the number of software test cases. The efficiency evaluation of the first phase focuses on the speed and effectiveness of converting resources into test cases, analyzing the rationality of environment setup and resource allocation. In the second phase, corresponding to the test case development and execution stage, the number of software test cases serves as input, while the outputs are the number of issues discovered and the average test execution time. The efficiency evaluation of the second phase emphasizes the quality and execution efficiency of test cases, assessing the output efficiency and issue detection efficiency of the testing process.

[0060] Furthermore, in this embodiment, when applying the network efficiency DEA model, a dummy variable is introduced as a coordinating variable. By setting the coordinating variable, it is ensured that the output of the first stage (the number of software test cases) can smoothly become the input of the second stage. The coordinating variable acts as a bridge in the model, ensuring the consistency and accuracy of the efficiency evaluation. Subsequently, the network efficiency DEA model calculates the efficiency values ​​of the first and second stages, as well as the corresponding weight parameters, by solving a linear programming problem. The efficiency value reflects the efficiency level of each team in its respective stage within a specific historical period, while the weight parameters include the degree of contribution of different input and output indicators to the overall efficiency.

[0061] Then, this embodiment utilizes the obtained first-stage efficiency value, first-stage weight parameter, second-stage efficiency value, and second-stage weight parameter to conduct an in-depth analysis of the automated testing efficiency values ​​across multiple preset historical time periods within a predetermined historical cycle. By comparing the efficiency values ​​across different time periods, peaks and troughs in efficiency can be identified, as well as the potential reasons for efficiency fluctuations. The weight parameters reveal the importance of each indicator in efficiency evaluation, helping to understand which inputs or outputs have the greatest impact on efficiency. For example, if a team's environment setup costs have a high weight while the number of testers has a low weight, it may mean that environment preparation is the bottleneck in efficiency, rather than a human factor. By comparing the efficiency values ​​across different historical time periods, this embodiment can track the trajectory of efficiency development and identify trends of efficiency improvement or decline.

[0062] Optionally, when using the network efficiency DEA model, the method further includes: for each development project team, ranking the efficiency values ​​of multiple development project teams using multiple automated test efficiency values ​​obtained from the network efficiency DEA model analysis, wherein the ranking methods include: overall efficiency value ranking, first-stage efficiency value ranking, and second-stage efficiency value ranking, and the overall efficiency value includes the first-stage efficiency value and the second-stage efficiency value; based on the efficiency value ranking results, determining the target stage in which the efficiency value decreases, wherein the target stage is the first stage or the second stage.

[0063] In this embodiment, the application of the network efficiency DEA model is not limited to calculating static efficiency. It further involves ranking the automated testing efficiency values ​​of different development project teams and determining the specific stages of efficiency decline based on the ranking results. This allows for precise identification of efficiency problems and the implementation of effective improvement measures. First, the model calculates the overall automated testing efficiency value for each team. This value combines the efficiency performance of the first and second stages, reflecting the team's efficiency level throughout the entire automated testing process. Then, all teams are ranked according to their overall efficiency value, with teams having higher efficiency values ​​ranked higher and those with lower values ​​ranked lower. This ranking method visually displays the comprehensive efficiency ranking of each team, facilitating the rapid identification of teams with excellent and insufficient efficiency. Second, the model also calculates and ranks the efficiency value of each team separately in the first stage (i.e., the environment setup and resource allocation stage). This method assesses the team's efficiency in environment preparation and resource allocation, identifying whether there are efficiency bottlenecks in this stage. Similarly, the model ranks the efficiency values ​​of all teams in the second stage (test case development and execution stage), deeply analyzing the efficiency of test design and execution to determine which teams excel or lag in the efficient development and rapid execution of test cases.

[0064] After ranking the efficiency values, this embodiment further analyzes the stages of efficiency decline. By comparing the overall efficiency value, the efficiency value of the first stage, and the efficiency value of the second stage, the stage with the most significant efficiency decline, i.e., the target stage, is identified. If a team's ranking in the overall efficiency ranking drops, but its ranking in the first stage remains stable, then the target stage of efficiency decline is determined to be the second stage. Conversely, if the team's ranking in the first stage drops, but its overall and second-stage rankings do not change significantly, then the target stage of efficiency decline is the first stage. This analytical method can accurately pinpoint the source of efficiency problems, whether it's environment setup and resource allocation, or test case development and execution.

[0065] Step S203: Based on the indicator system model, use the dynamic Malmquist index to analyze the dynamic changes in the automated testing efficiency value and output a testing efficiency evaluation report. The testing efficiency report includes at least: the automated testing efficiency value of the development project team in each historical time period and the dynamic efficiency change trend in the automated testing implementation process in the predetermined historical time period.

[0066] In this embodiment, it is necessary to further analyze the dynamic changes of the automated testing efficiency value and generate a test efficiency evaluation report. This report can reflect the automated testing efficiency performance of each development project team in different historical time periods, as well as the trend of efficiency changes. This involves obtaining time-series test panel data, using the dynamic Malmquist index for analysis, and finally outputting a comprehensive evaluation report.

[0067] Optionally, the step of using the dynamic Malmquist index to analyze the dynamic changes in the efficiency value of automated testing and outputting a test efficiency evaluation report includes: obtaining test panel data containing time series; using the dynamic Malmquist index to analyze the test panel data to obtain the software project development technology efficiency parameters and efficiency fluctuation parameters containing time series during the automated testing process; and outputting a test efficiency evaluation report based on the software project development technology efficiency parameters and efficiency fluctuation parameters in the time series.

[0068] First, this embodiment can collect test panel data containing time series, which not only records the automated testing efficiency values ​​of each development project team, but also covers the efficiency changes in different historical time periods, providing a solid data foundation for dynamic analysis. The use of panel data enables the analysis to comprehensively examine the automated testing efficiency from both cross-sectional and time series dimensions.

[0069] Next, this embodiment uses the dynamic Malmquist index method to conduct in-depth analysis of the test panel data. The Malmquist index includes technical efficiency parameters and efficiency fluctuation parameters. The technical efficiency parameters reflect the horizontal comparison of the efficiency of each team under the same technical conditions; while the efficiency fluctuation parameters reveal the vertical changes in efficiency over time. Specifically, through dynamic Malmquist index analysis, this embodiment can calculate the technical efficiency level of each development project team in different historical time periods, identify which teams maintained or improved their technical efficiency during the implementation of automated testing, and which teams experienced a decline in technical efficiency, providing a clear direction for subsequent technical optimization. The efficiency fluctuation parameters reveal the trend of efficiency changes over time, including increases or decreases in technical efficiency, as well as efficiency changes due to management or process improvements. By comparing the efficiency fluctuation parameters at different times, this embodiment can understand the overall direction of changes in automated testing efficiency and identify key factors for efficiency improvement.

[0070] This embodiment integrates time-series software project development technical efficiency parameters and efficiency fluctuation parameters to generate a test efficiency evaluation report. The report details the automated testing efficiency values ​​of each development project team at different historical time periods, provides quantitative indicators of efficiency performance, and, combined with the dynamic Malmquist index analysis results, depicts the dynamic trend of automated testing efficiency within a predetermined historical time period, including the improvement and fluctuation of technical efficiency, as well as the improvement of efficiency management.

[0071] Optionally, the steps of using dynamic Malmquist index analysis to analyze test panel data and obtain time-series software project development technology efficiency parameters and efficiency fluctuation parameters during automated testing include: using dynamic Malmquist index analysis to obtain a dynamic index set; decomposing the dynamic index set into time-series software project development technology efficiency parameters and software project development technology progress parameters; and determining the efficiency fluctuation parameters of the automated testing process at different historical time points based on the decomposed software project development technology efficiency parameters and software project development technology progress parameters.

[0072] In this embodiment, the application of the dynamic Malmquist index is not limited to the evaluation of static efficiency. Furthermore, it can analyze the dynamic changes in efficiency over time during automated testing, providing in-depth insights into technical efficiency and efficiency fluctuations. First, this embodiment applies the dynamic Malmquist index to collected test panel data, which includes efficiency performance data sets from multiple development project teams at different points in time. The calculation of the dynamic Malmquist index aims to capture long-term trends in efficiency, revealing the complex relationship between technical efficiency and efficiency fluctuations. After applying the dynamic Malmquist index, a series of dynamic indices are generated, forming a dynamic index set. These dynamic indices reflect the changes in automated testing efficiency at different historical points in time, laying the foundation for subsequent efficiency fluctuation parameter analysis.

[0073] This embodiment further decomposes the dynamic index set into two sets of key parameters: software project development technology efficiency parameters and software project development technology progress parameters. The former measures the efficiency of automated testing at a given level of technology, i.e., whether the team can achieve maximum output with minimum input; the latter examines the impact of changes in technology itself on testing efficiency, assessing whether technological progress has promoted efficiency improvement. Finally, based on the decomposed software project development technology efficiency parameters and software project development technology progress parameters, this embodiment can determine the efficiency fluctuation parameters of the automated testing process at different historical points in time.

[0074] Optionally, the steps for outputting a test efficiency evaluation report based on the software project development technical efficiency parameters and efficiency fluctuation parameters over time include: retrieving the automated testing efficiency values ​​of each development project team involved in the target software project in different preset historical time periods obtained after evaluating static efficiency using the network efficiency DEA model; retrieving the efficiency fluctuation parameters of the automated testing process at different historical time points using the dynamic Malmquist index analysis; unifying the time axis of the static efficiency evaluation using the network efficiency DEA model and the dynamic changes of the automated testing efficiency values ​​using the dynamic Malmquist index analysis, to obtain a test efficiency evaluation report that includes the automated testing efficiency values ​​and dynamic efficiency change trends for each historical time period.

[0075] First, this embodiment retrieves the automated testing efficiency values ​​of each development team involved in the target software project during different preset historical time periods, obtained when evaluating static efficiency using the network efficiency DEA model. These efficiency values ​​reflect the team's efficiency level at specific points in time, providing static efficiency data support for the subsequent report generation. Next, this embodiment retrieves efficiency fluctuation parameters obtained by analyzing the automated testing process at different historical points in time using the dynamic Malmquist index. These parameters reveal the trend of efficiency changes over time, including efficiency increases, decreases, or stabilization, adding a dynamic efficiency change perspective to the report. Finally, this embodiment unifies the timeline of static efficiency evaluation using the network efficiency DEA model with the timeline of efficiency fluctuation analysis using the dynamic Malmquist index, ensuring data consistency and comparability, so that the report accurately reflects the overall picture of automated testing efficiency.

[0076] By comprehensively analyzing data on a unified timeline, a test efficiency evaluation report is generated. The report can display in detail the automated test efficiency value for each historical time period, as well as the dynamic trend of efficiency value over time. This comprehensive efficiency view not only reflects the static state of efficiency, but also captures the dynamic evolution of efficiency.

[0077] Optionally, after outputting the test efficiency evaluation report based on the software project development technical efficiency parameters and efficiency fluctuation parameters over time, the report also includes: analyzing the target teams in each development project team that have experienced efficiency decline based on the automated test efficiency values ​​and dynamic efficiency change trends for each historical time period in the test efficiency evaluation report; querying the contact information of the team leaders of the target teams; and outputting automated test efficiency decline prompts and automated test optimization suggestions to the contact information.

[0078] After outputting the test efficiency evaluation report, this embodiment further analyzes the target team experiencing efficiency decline and provides alerts and optimization suggestions regarding the decline in automated testing efficiency to the team leaders within the target team. Based on the dynamic efficiency trends in the report, this embodiment identifies the team experiencing efficiency decline, i.e., the target team. By comparing efficiency values ​​and efficiency fluctuation parameters at different times, the timing and magnitude of the efficiency decline can be precisely pinpointed. After identifying the target team, this embodiment retrieves the contact information for the team leaders within the target team. Subsequently, it sends early warning messages about the decline in automated testing efficiency, along with optimization suggestions based on the efficiency analysis results, to these contact individuals. These optimization suggestions can cover multiple aspects, including resource allocation, technology upgrades, team training, and test process improvements, aiming to help the target team identify the root causes of the problem and take effective measures to improve testing efficiency.

[0079] Through the above steps, a pre-built indicator system model can be invoked. This model includes at least: test input indicators, intermediate products, and test output indicators. Test input indicators include: environment setup costs and the number of testers. Intermediate products include: the number of software test cases. Test output indicators include: the number of issues found and the average test execution time. Based on the indicator system model, the network efficiency DEA model is used to evaluate static efficiency, analyzing the automated testing efficiency values ​​of each development project team involved in the target software project across multiple preset historical time periods within a predetermined historical period. Based on the indicator system model, the dynamic Malmquist index is used to analyze the dynamic changes in automated testing efficiency values, outputting a test efficiency evaluation report. This report includes at least: the automated testing efficiency values ​​of the development project teams in each historical time period, and the dynamic efficiency change trend during the automated testing implementation process within the predetermined historical time period. In this embodiment, a comprehensive test efficiency evaluation report can be generated by static efficiency assessment of the network DEA and time series analysis of the dynamic Malmquist index. By utilizing the dynamic Malmquist index, this invention can monitor the long-term trend of automated testing efficiency, comprehensively evaluate automated testing efficiency from both static and dynamic dimensions, provide multi-angle efficiency analysis results, and greatly improve the accuracy of the evaluation. This solves the technical problem of low accuracy in evaluating the efficiency of automated testing of software projects, which relies on qualitative or semi-quantitative indicators.

[0080] The following describes in detail another optional implementation method.

[0081] This invention employs a network DEA method combined with a dynamic Malmquist exponent model, integrating static and dynamic perspectives, to analyze the automation testing efficiency of multiple teams during the automation implementation cycle.

[0082] First, efficiency analysis indicators were selected from both input and output dimensions, and objective data corresponding to these indicators were collected. Second, the network DEA method was used to analyze the automation efficiency of different teams at a certain point in time through static cross-sectional results. Third, the dynamic Malmquist index model was used to evaluate the overall trend of different teams in the process of implementing automated testing through dynamic longitudinal results across different periods.

[0083] The overall concept of the embodiments of the present invention is as follows:

[0084] (1) Construct an indicator system model. First, start from the input and output dimensions of the field of automated testing, deeply explore the key elements, and construct an indicator system model.

[0085] (2) Using the network DEA method, the automation testing efficiency of different teams at a certain time point was obtained. The results include the overall efficiency value. and the efficiency value of the first stage Second-stage efficiency value The results data can be analyzed to determine the automation testing efficiency of each team, allowing for a ranking of teams based on their efficiency. Furthermore, the efficiency of each stage can be analyzed to identify which specific stages require optimization.

[0086] (3) The dynamic Malmquist method is used to evaluate the changes in the team's automated testing efficiency in different periods through dynamic longitudinal results across different time periods. The analysis results can be interpreted to see which teams have improved efficiency and which teams have declined efficiency in different time periods, so that targeted efficiency improvements can be made.

[0087] It should be noted that the indicators selected in the embodiments of the present invention are all quantitative indicators, and the analysis results are more reliable, thereby making the evaluation of the efficiency of automated testing more realistic and credible.

[0088] The following describes the detailed implementation of the embodiments of the present invention in conjunction with the overall concept.

[0089] The first part is to construct the model indicator system.

[0090] The indicator system for analyzing the efficiency of automated testing is constructed based on the principles of scientific rigor, systematic approach, and domain-specific focus, and is set from three dimensions: input, intermediate products, and output. Specific attribute descriptions are as follows:

[0091] (1) Input indicators: X1 - Environment setup cost, X2 - Human resources.

[0092] (2) Intermediate product: Z - number of test cases.

[0093] (2) Output indicators: Y1 - number of problems found, Y2 - average test execution time.

[0094] The indicators are interpreted as follows: (1) In the input indicator dimension, “environment setup cost” refers to the cost required to build an automated framework (such as software costs such as framework copyright fees, technical consulting fees, and hardware equipment costs such as servers and test hosts), and “human resources” refers to the number of personnel invested in automated testing; (2) In the intermediate product dimension, the number of test cases refers to the number of test cases run by automated testing; (3) In the output indicator dimension, the number of problems found refers to the number of problems found through automated testing, and the average test execution time refers to the average time required to execute a full-process automated regression test.

[0095] Part Two: Introduction to the Principles and Calculation Steps of the Network DEA Model.

[0096] Network DEA dissects the "black box" of decision-making units, viewing their internal operations as a network of interconnected subprocesses, particularly focusing on two consecutive stages and their key intermediate products. Its core advantage lies in its ability to reveal internal efficiency structures, accurately pinpoint inefficient processes, rationally handle intermediate products, and analyze relationships between stages. This provides more in-depth, accurate, and actionable efficiency assessment results than traditional DEA, offering strong support for managers' decision-making.

[0097] This model decomposes the internal structure of the decision-making unit (DMU) into two related sub-processes (Phase 1 and Phase 2), and simultaneously evaluates the overall efficiency and the efficiency of each sub-process. Assume there are n DMUs, each containing two phases:

[0098] Phase 1: Consume external input X i Production of intermediate product Z d .

[0099] Phase 2: Consuming intermediate product Z d Production final output Y r .

[0100] Key characteristic: Intermediate product Z d It is both the output of stage 1 and the input of stage 2.

[0101] The symbols are defined as follows:

[0102] X ik : The i-th type of input to the k-th DMU (i = 1, ..., m).

[0103] Z dk : The quantity of the d-th intermediate product in the k-th DMU (d=1,...,D).

[0104] Y rk : The r-th output of the k-th DMU (r = 1, ..., s).

[0105] Decision variables:

[0106] θ: Overall efficiency value (scalar, 0 < θ ≤ 1).

[0107] λ_k∧{(1)}: The weights for the k-th DMU in the first stage.

[0108] λ_k∧{(2)}: The weights for the k-th DMU in the second stage.

[0109] Z_d∧*: Virtual intermediate product (d=1,...,D), serving as a variable connecting the two stages.

[0110] The mathematical calculation formulas used in this embodiment for the network DEA model include:

[0111] Mathematical model: minθ o

[0112]

[0113] Among them, 1. Formula 1 and Formula 2 belong to Stage 1 constraints (minimize input); 2. Formula 3 and Formula 4 belong to Stage 2 constraints (maximize output);

[0114] 3. Formula 5 belongs to the consistency constraint of intermediate products;

[0115] 4. Formula 6 is a non-negativity constraint;

[0116] 5. Formula 7 is a range constraint.

[0117] Furthermore, let me explain in detail the steps of using the DEA network for computation.

[0118] 1. Problem initialization.

[0119] Selected DMUs to be evaluated o Set its input X io Output Y ro and intermediate product Z do .

[0120] 2. Construct a planning model.

[0121] Objective function: Minimize the overall investment ratio θ o .

[0122] Phase 1 constraint: Ensure virtual input ≤ θ o X io Virtual intermediate output ≥ Phase 2 constraint: Ensure virtual intermediate consumption ≤ Virtual final output ≥ Y ro Introducing dummy variables As a coordinating variable between the two stages.

[0123] 3. Solve the linear programming problem.

[0124] Solving the above mathematical formula yields the optimal solution:

[0125] (1) Overall efficiency value

[0126] (2) Phase 1 weights Phase 2 weights

[0127] (3) Optimal intermediate product value

[0128] 4. Decomposition sub-stage efficiency.

[0129] First-stage efficiency value: (Because the model is input-oriented and the first stage only has input constraints, the efficiency of the first stage can be directly taken as the overall efficiency.) ).

[0130] Second-stage efficiency value: The efficiency is calculated using the following formula (with the second-stage output-oriented calculation method):

[0131]

[0132] Furthermore, the following evaluation criteria are provided, including:

[0133] 1. Overall efficiency.

[0134] Overall efficiency value A higher value indicates higher efficiency. This indicates the percentage of input that can be reduced to achieve optimal efficiency. For example... This indicates that in order to achieve optimal efficiency, input can be reduced by 20%.

[0135] The overall efficiency value of each DMU is obtained from the calculation results. This allows us to sort the DMUs and obtain a ranking of their efficiency.

[0136] 2. Stage efficiency.

[0137] First-stage efficiency value: A higher value indicates higher efficiency in the first stage.

[0138] Second-stage efficiency value: A higher value indicates higher efficiency in the second stage.

[0139] Diagnostic rules can be specifically divided into the following four scenarios:

[0140] Scene 1:

[0141] Meaning: The first stage is not optimal, but the second stage is optimal;

[0142] Areas for improvement: Optimize the investment in the first phase.

[0143] Scene 2:

[0144] Meaning: The first stage is optimal, but the second stage is not optimal;

[0145] Areas for improvement: Optimize the process in the second phase.

[0146] Scene 3:

[0147] Meaning: The first stage is not optimal, and the second stage is not optimal;

[0148] Improvement direction: Full process restructuring.

[0149] Scene 4:

[0150] Meaning: The first stage is optimal, and the second stage is optimal;

[0151] Areas for improvement: The overall process efficiency is already optimal.

[0152] Part Three: Introduction to the Principles and Calculation Steps of the Dynamic Malmquist Method.

[0153] The dynamic Malmquist index method extends the traditional Malmquist index method by considering changes over time. It doesn't just compare two points in time, but analyzes trends over the entire period, while retaining the advantages of the traditional Malmquist method in efficiency evaluation. The traditional Malmquist index measures the efficiency changes of decision-making units across different periods from a dynamic perspective. Furthermore, the Malmquist index can be further decomposed into technical efficiency and technical progress, reflecting changes in technology and efficiency, thus providing a more accurate reflection of dynamic efficiency changes.

[0154] The Dynamic Malmquist Index (DMIP) can analyze efficiency changes from period 1 to period t+1. The Dynamic Malmquist Index can be further decomposed into technical efficiency (EC) and technological progress (TC), with the following relationship: Dynamic Malmquist Index (DMIP) = Technical Efficiency (EC) × Technological Progress (TC).

[0155] The meanings of the above indicator values ​​are as follows:

[0156] Among them, the Malmquist index MI: MI>1 indicates that productivity increases from period t to period t+1, and vice versa.

[0157] Technical efficiency (EC): EC>1 indicates improved technical efficiency, while EC>1 indicates decreased technical efficiency.

[0158] Technological progress (TC): TC>1 indicates technological progress, and TC>1 indicates technological decline.

[0159] Furthermore, the calculation steps for the dynamic Malmquist exponent are provided below.

[0160] 1. Calculation method of dynamic Malmquist index DMPI.

[0161] Suppose there are Malmquist exponents M1, M2, ..., M1 over T-1 periods. T-1 The formula for calculating the dynamic Malmquist index (DMPI) is as follows:

[0162] DMPI = MPI1 × MPI2 × ... × MPI T-1 ;

[0163] The formulas for calculating technical efficiency (EC) and technological progress (TC) are as follows:

[0164] EC = EC1 × EC2 × ... × EC T-1 ;

[0165] TC = TC1 × TC2 × ... × TC T-1 .

[0166] 2. Malmquist Index (MPI) for each period c Calculation method

[0167] The Malmquist index for each period is calculated using the following formula:

[0168]

[0169] Among them, (1)(x t y t () represents the input and output in period t;

[0170] (2)(x t+1 y t+1 (t+1) represents the input and output at time (t+1);

[0171] (3) Let represent the distance function of the c-th decision-making unit at time t, with the technological level T at time t as a reference;

[0172] (4) Let represent the distance function of the c-th decision-making unit in period (t+1) with reference to the technological level T in period t.

[0173] (5) Let represent the distance function of the c-th decision-making unit at time t, with the technological level T at time (t+1) as a reference;

[0174] (6) Let represent the distance function of the c-th decision-making unit at time (t+1) with reference to the technological level T at time (t+1).

[0175] MPC c The index can be further decomposed into technical efficiency (EC).c ) and technological progress (TC c The specific breakdown is as follows:

[0176] Thus, EC is derived c and TC c Calculation method:

[0177]

[0178] This invention employs a network DEA model and a dynamic Malmquist index to provide a method suitable for efficiency analysis in automated testing. The efficiency analysis method used in this invention is objective, effective, and highly reliable, focusing more on reflecting the inherent statistical regularities of the data itself. It analyzes input and output indicators from both static and dynamic dimensions, overcoming the problems of existing efficiency evaluation methods that fail to fully reflect the potential information in the data, lack comprehensive evaluation standards, and have insufficient analytical dimensions. This makes the efficiency analysis results more reliable and accurate, thus providing technical support for subsequent efficiency optimization.

[0179] The following is a detailed description with reference to another embodiment.

[0180] Example 2

[0181] The software project automated testing efficiency analysis device provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in the above embodiment one. Its specific implementation method and beneficial effects can be referred to the foregoing method embodiment, and will not be repeated here.

[0182] Figure 3 This is a schematic diagram of an optional software project automated testing efficiency analysis device according to an embodiment of the present invention, such as... Figure 3 As shown, the analysis device for the automated testing efficiency of the software project may include: an indicator system model calling unit 31, a static testing efficiency evaluation unit 32, and a dynamic testing change analysis unit 33.

[0183] The indicator system model calling unit 31 is used to call the pre-built indicator system model. The indicator system model includes at least: test input indicators, intermediate products and test output indicators. Test input indicators include: environment setup cost and number of testers. Intermediate products include: number of software test cases. Test output indicators include: number of problems found and average test execution time.

[0184] The static efficiency evaluation unit 32 is used to evaluate static efficiency based on the index system model and the network efficiency DEA model, and to analyze the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period.

[0185] The dynamic change analysis unit 33 is used to analyze the dynamic changes of automated test efficiency values ​​based on the indicator system model and the dynamic Malmquist index, and output a test efficiency evaluation report. The test efficiency report includes at least: the automated test efficiency values ​​of the development project team in each historical time period, and the dynamic efficiency change trend in the automated test implementation process in the predetermined historical time period.

[0186] The aforementioned software project automated testing efficiency analysis device can invoke a pre-built indicator system model through indicator system model invocation unit 31. The indicator system model includes at least: test input indicators, intermediate products, and test output indicators. Test input indicators include: environment setup costs and the number of testers. Intermediate products include: the number of software test cases. Test output indicators include: the number of problems found and the average test execution time. Based on the indicator system model, the static efficiency evaluation unit 32 uses the network efficiency DEA model to evaluate static efficiency, analyzing the automated testing efficiency values ​​of each development team involved in the target software project during multiple preset historical time periods within a predetermined historical period. Based on the indicator system model, the dynamic change analysis unit 33 uses the dynamic Malmquist index to analyze the dynamic changes in automated testing efficiency values ​​and outputs a test efficiency evaluation report. The test efficiency report includes at least: the automated testing efficiency values ​​of the development team in each historical time period and the dynamic efficiency change trend during the automated testing implementation process within the predetermined historical time period. In this embodiment, a comprehensive test efficiency evaluation report can be generated by static efficiency assessment of the network DEA and time series analysis of the dynamic Malmquist index. By utilizing the dynamic Malmquist index, this invention can monitor the long-term trend of automated testing efficiency, comprehensively evaluate automated testing efficiency from both static and dynamic dimensions, provide multi-angle efficiency analysis results, and greatly improve the accuracy of the evaluation. This solves the technical problem of low accuracy in evaluating the efficiency of automated testing of software projects, which relies on qualitative or semi-quantitative indicators.

[0187] Optionally, the static efficiency evaluation unit includes: a first decomposition module, used to decompose the automated testing process of the software project into a first phase and a second phase, wherein the first phase is the software project environment setup and resource configuration phase, and the second phase is the test case development and execution phase; a phase generation module, used to generate the number of software test cases in the first phase based on the environment setup cost and the number of testers, and to generate the number of problems found and the average test execution time in the second phase based on the number of software test cases; a dummy variable usage module, used to introduce predetermined dummy variables, and to use the network efficiency DEA model to use the predetermined dummy variables as coordination variables between the first and second phases to calculate the efficiency values ​​of the first and second phases; and a first analysis module, used to analyze the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period, based on the efficiency values ​​of the first phase and the corresponding weight parameters of the first phase, and the efficiency values ​​of the second phase and the corresponding weight parameters of the second phase.

[0188] Optionally, when using the network efficiency DEA model, the static efficiency evaluation unit further includes: a first sorting module, used to sort the efficiency values ​​of multiple development project teams using multiple automated test efficiency values ​​obtained from the network efficiency DEA model analysis for each development project team, wherein the sorting methods include: overall efficiency value sorting, first-stage efficiency value sorting, and second-stage efficiency value sorting, and the overall efficiency value includes the first-stage efficiency value and the second-stage efficiency value; and an efficiency decline analysis module, used to determine the target stage in which the efficiency value decline occurs based on the efficiency value sorting results, wherein the target stage is the first stage or the second stage.

[0189] Optionally, the test dynamic change analysis unit includes: a test panel data acquisition module for acquiring test panel data containing time series; a data analysis module for analyzing the test panel data using the dynamic Malmquist index to obtain software project development technical efficiency parameters and efficiency fluctuation parameters containing time series during the automated testing process; and a report output module for outputting a test efficiency evaluation report based on the software project development technical efficiency parameters and efficiency fluctuation parameters in the time series.

[0190] Optionally, the data analysis module includes: a data analysis submodule for analyzing test panel data using the dynamic Malmquist index to obtain a dynamic index set; a second decomposition module for decomposing the dynamic index set into software project development technology efficiency parameters and software project development technology progress parameters containing time series data; and a parameter determination submodule for determining the efficiency fluctuation parameters of the automated testing process at different historical time points based on the decomposed software project development technology efficiency parameters and software project development technology progress parameters.

[0191] Optionally, the report output module includes: a first retrieval submodule, used to retrieve the automated testing efficiency values ​​of each development project team involved in the target software project in different preset historical time periods after evaluating static efficiency using the network efficiency DEA model; a second retrieval submodule, used to retrieve the efficiency fluctuation parameters of the automated testing process at different historical time points using the dynamic Malmquist index analysis; and a time axis unification submodule, used to unify the time axis of the dynamic changes of the automated testing efficiency values ​​evaluated using the network efficiency DEA model and the dynamic changes of the automated testing efficiency values ​​analyzed using the dynamic Malmquist index, to obtain a test efficiency evaluation report that includes the automated testing efficiency values ​​and dynamic efficiency change trends for each historical time period.

[0192] Optionally, the software project automated testing efficiency analysis device further includes: a team analysis unit, used to analyze the target teams in each development project team that have experienced efficiency decline based on the automated testing efficiency values ​​and dynamic efficiency change trends of each historical time period in the test efficiency evaluation report after outputting the test efficiency evaluation report according to the software project development technical efficiency parameters and efficiency fluctuation parameters over a time series; and an optimization suggestion unit, used to query the contact information of the team responsible for the target team, and output automated testing efficiency decline prompts and automated testing optimization suggestions to the contact information.

[0193] The aforementioned software project automated testing efficiency analysis device may also include a processor and a memory. The aforementioned indicator system model calling unit 31, test static efficiency evaluation unit 32, test dynamic change analysis unit 33, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0194] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and by adjusting kernel parameters, automated testing efficiency analysis based on the network DEA model and the Malmquist index can be achieved.

[0195] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0196] Example 3

[0197] Embodiments of this application may provide an electronic device. Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application, which provides a method for analyzing the efficiency of automated testing of software projects. Figure 4As shown, the electronic device may include: one or more ( Figure 4 Only one of the following is shown: processor 402, memory 404, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0198] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the software project automated testing efficiency analysis method and apparatus in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned software project automated testing efficiency analysis method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0199] The processor can access information and applications stored in memory via a transmission device to execute the following steps: Invoking a pre-built indicator system model, wherein the indicator system model includes at least: test input indicators, intermediate products, and test output indicators. Test input indicators include: environment setup costs and the number of testers; intermediate products include: the number of software test cases; and test output indicators include: the number of issues found and the average test execution time. Based on the indicator system model, using a network efficiency DEA model to evaluate static efficiency, and analyzing the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period. Based on the indicator system model, using a dynamic Malmquist index to analyze the dynamic changes in automated testing efficiency values, and outputting a test efficiency evaluation report, wherein the test efficiency report includes at least: the automated testing efficiency values ​​of the development project teams in each historical time period, and the dynamic efficiency change trend during the automated testing implementation process in the predetermined historical time period.

[0200] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0201] Those skilled in the art will understand that all or part of the steps in the various software project automated testing efficiency analysis methods of the above embodiments can be completed by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0202] Example 4

[0203] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the software project automated testing efficiency analysis method provided in Embodiment 1.

[0204] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is running, the device where the computer-readable storage medium is located executes the software project automated testing efficiency analysis method of any one of the above embodiments.

[0205] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0206] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method for analyzing the efficiency of automated testing of software projects as described in various embodiments of this application.

[0207] This application also provides a computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the software project automated testing efficiency analysis method described in various embodiments of this application.

[0208] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0209] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0210] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0211] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0212] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0213] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0214] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for analyzing the efficiency of automated testing in software projects, characterized in that, include: The system calls a pre-built indicator model, which includes at least: test input indicators, intermediate products and test output indicators. The test input indicators include: environment setup cost and number of testers. The intermediate products include: number of software test cases. The test output indicators include: number of problems found and average test execution time. Based on the aforementioned indicator system model, the static efficiency is evaluated using the network efficiency DEA model, and the automated testing efficiency values ​​of each development project team involved in the target software project are analyzed in multiple preset historical time periods within a predetermined historical period. Based on the aforementioned indicator system model, the dynamic changes in the automated testing efficiency value are analyzed using the dynamic Malmquist index, and a testing efficiency evaluation report is output. The testing efficiency report includes at least: the automated testing efficiency value of the development project team in each historical time period, and the dynamic efficiency change trend during the implementation of automated testing in a predetermined historical time period.

2. The analytical method according to claim 1, characterized in that, The steps for evaluating static efficiency using the network efficiency DEA model include: The automated testing process for a software project is broken down into two phases: the first phase is the setup and resource configuration of the software project environment, and the second phase is the development and execution of test cases. The network efficiency DEA model is used to generate the number of software test cases in the first stage based on the cost of setting up the environment and the number of testers, and in the second stage, the number of problems found and the average test execution time are generated based on the number of software test cases. A predetermined dummy variable is introduced, and the network efficiency DEA model is used to use the predetermined dummy variable as a coordination variable between the first stage and the second stage to calculate the efficiency value of the first stage and the efficiency value of the second stage. Based on the efficiency value of the first stage and the corresponding weight parameters of the first stage, the efficiency value of the second stage and the corresponding weight parameters of the second stage, the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period are analyzed.

3. The analytical method according to claim 2, characterized in that, When using the network efficiency DEA model, it also includes: For each development project team, the multiple automated test efficiency values ​​obtained from the network efficiency DEA model are used to rank the efficiency values ​​of the multiple development project teams. The ranking methods include: overall efficiency value ranking, first-stage efficiency value ranking, and second-stage efficiency value ranking. The overall efficiency value includes the first-stage efficiency value and the second-stage efficiency value. Based on the efficiency value ranking results, the target stage in which the efficiency value decreases is determined, wherein the target stage is either the first stage or the second stage.

4. The analytical method according to claim 1, characterized in that, The steps for analyzing the dynamic changes in the automated testing efficiency value using the dynamic Malmquist index and outputting a testing efficiency evaluation report include: Obtain test panel data containing time series data; The test panel data was analyzed using the dynamic Malmquist index to obtain the software project development technology efficiency parameters and efficiency fluctuation parameters containing time series data during the automated testing process. Based on the software project development technology efficiency parameters and efficiency fluctuation parameters of the time series, the test efficiency evaluation report is output.

5. The analytical method according to claim 4, characterized in that, The steps for analyzing the test panel data using the dynamic Malmquist index to obtain time-series software project development technical efficiency parameters and efficiency fluctuation parameters during automated testing include: The test panel data was analyzed using the dynamic Malmquist index to obtain a dynamic index set; The dynamic index set is decomposed into software project development technology efficiency parameters and software project development technology progress parameters, which contain time series data. Based on the decomposed software project development technology efficiency parameters and software project development technology progress parameters, the efficiency fluctuation parameters of the automated testing process at different historical points in time are determined.

6. The analytical method according to claim 4, characterized in that, The steps for outputting the test efficiency evaluation report based on the software project development technical efficiency parameters and efficiency fluctuation parameters of the time series include: Retrieve the automated testing efficiency values ​​of each development team involved in the target software project during different preset historical time periods after evaluating static efficiency using the network efficiency DEA model. Retrieve and analyze efficiency fluctuation parameters of the automated testing process at different historical points in time using the dynamic Malmquist index; By unifying the timeline for evaluating static efficiency using the network efficiency DEA model and analyzing the dynamic changes of the automated test efficiency value using the dynamic Malmquist index, a test efficiency evaluation report is obtained that includes the automated test efficiency value for each historical time period and the dynamic efficiency change trend.

7. The analytical method according to claim 6, characterized in that, After outputting the test efficiency evaluation report based on the software project development technical efficiency parameters and efficiency fluctuation parameters of the time series, the report also includes: Based on the automated testing efficiency values ​​and dynamic efficiency trends for each historical time period in the aforementioned test efficiency evaluation report, the target teams experiencing efficiency decline in each development project team are analyzed. Query the contact information of the team leader in charge of the target team, and output the automated testing efficiency degradation prompt and automated testing optimization suggestions to the contact information.

8. A device for analyzing the efficiency of automated testing in software projects, characterized in that, include: The indicator system model invocation unit is used to invoke a pre-built indicator system model, wherein the indicator system model includes at least: test input indicators, intermediate products and test output indicators, the test input indicators include: environment setup cost and number of testers, the intermediate products include: number of software test cases, and the test output indicators include: number of problems found and average test execution time. The static efficiency evaluation unit is used to evaluate static efficiency based on the index system model and the network efficiency DEA model, and to analyze the automated testing efficiency values ​​of each development project team involved in the target software project in multiple preset historical time periods within a predetermined historical period. The test dynamic change analysis unit is used to analyze the dynamic changes of the automated test efficiency value based on the indicator system model and the dynamic Malmquist index, and output a test efficiency evaluation report. The test efficiency report includes at least: the automated test efficiency value of the development project team in each historical time period and the dynamic efficiency change trend in the automated test implementation process in a predetermined historical time period.

9. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method for analyzing the efficiency of automated testing of software projects as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for analyzing the efficiency of automated testing of software projects as described in any one of claims 1 to 7.

Images