A method for evaluating R&D work efficiency and a storage medium

Abstract

This application provides a method and storage medium for evaluating R&D work efficiency, relating to the field of data processing technology, for accurately evaluating the efficiency of R&D work. The method includes: in response to an evaluation instruction, acquiring multiple target evaluation indicators and target R&D coefficients; the R&D tasks include: deleting, modifying, and / or adding target functions of the target software. The multiple target evaluation indicators include: target code coverage, unit test case effectiveness, and technical debt repair rate. Based on the multiple target evaluation indicators, an initial indicator score is determined for each target evaluation indicator. Based on the multiple initial indicator scores and the target R&D coefficients, an R&D evaluation result is determined, which is used to indicate the efficiency of the target R&D team in completing the R&D tasks.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for evaluating the efficiency of research and development work and a storage medium. Background Technology

[0002] In the software development process, R&D efficiency evaluation systems can interface with software development-related systems to obtain indicator data during the R&D process. Based on this data, the system can evaluate the work efficiency of the R&D team, allowing it to assign R&D tasks to different teams according to the evaluation results. However, this method typically assigns a fixed weight to each evaluation indicator in a one-time manner, making it difficult to reflect the true R&D efficiency under different scenarios, resulting in low accuracy in the evaluation results of R&D team efficiency.

[0003] Therefore, improving the accuracy of the evaluation results of R&D team efficiency has become an urgent technical problem to be solved. Summary of the Invention

[0004] The purpose of this application is to provide a method and storage medium for evaluating the efficiency of research and development work, aiming to solve the problem of how to accurately evaluate the efficiency of research and development work.

[0005] To achieve the above objectives, firstly, this application provides a method for evaluating R&D work efficiency. The method includes: responding to an evaluation instruction, acquiring multiple target evaluation indicators and a target R&D coefficient. The multiple target evaluation indicators are used to evaluate the efficiency of a target R&D team in completing R&D tasks, which include: deleting, modifying, and / or adding target functions to the target software. The multiple target evaluation indicators include: target code coverage, unit test case effectiveness, and technical debt repair rate. The target code is used to implement the target functions. The target coverage rate indicates the degree to which unit test cases cover the target code. The technical debt repair rate is the ratio between the number of repaired technical debts and the number of previous technical debts. The number of repaired issues is the difference between the number of previous technical debts and the number of current technical debts. Previous technical debts are those detected by static code testing of historical code, and current technical debts are those detected by static code testing of the target code. The target code is the updated historical code. The target R&D coefficient reflects the complexity of the R&D task. Based on the multiple target evaluation indicators, an initial indicator score is determined for each target evaluation indicator. Based on multiple initial indicator scores and target R&D coefficients, the R&D evaluation results are determined, and these results are used to indicate the efficiency of the target R&D team in completing R&D tasks.

[0006] The solution provided in this application, in response to an evaluation instruction, obtains the target unit test coverage, unit test case validity, and technical debt repair rate of the target code corresponding to the target software, and determines the corresponding initial indicator scores. This allows for the determination of the coverage of unit test cases to the target code, the validity of unit test cases, and the status of technical debt repair, thereby accurately determining the completion status of the target R&D team for R&D tasks such as deleting, modifying, and / or adding target functions to the target software, and thus determining the work efficiency of the target R&D team. Simultaneously, by obtaining the target R&D coefficient, the complexity of the R&D tasks can be considered, thereby accurately determining the R&D evaluation results and achieving an accurate assessment of the R&D work efficiency of the target R&D team.

[0007] In some embodiments, there are multiple target R&D teams, and the target R&D coefficient is obtained as follows: An initial R&D coefficient is determined for each target R&D team. This initial R&D coefficient is related to the amount of code in the target software provided by the target R&D team and the number of target functions corresponding to the completion of the R&D task. The initial R&D coefficients corresponding to multiple target R&D teams are monotonically sorted. The sorted initial R&D coefficients are then grouped to obtain multiple monotonically sorted sets of initial R&D coefficients. Based on the index of each set of initial R&D coefficients, the target R&D coefficient corresponding to that set is determined.

[0008] In some embodiments, the initial R&D coefficient is obtained by: acquiring the number of completed target functions, a first average code size, a second average code size, the number of adjusted functions, and the predicted number of completed functions. The first average code size is the average code size of the target R&D team, the second average code size is the average code size of the target R&D department to which the target R&D team belongs, the number of adjusted functions is the number of target functions for which code needs to be added and / or modified, and the predicted number of functions is the expected number of target functions to be completed. The initial R&D coefficient is determined based on the number of completed target functions, the first average code size, the second average code size, the number of adjusted functions, and the predicted number of completed functions.

[0009] In some embodiments, the initial R&D coefficients satisfy the following formula: ; in, The initial R&D coefficient is represented by N1, the number of target functions completed is represented by Lavg1, the first average code size is represented by Lavg2, the second average code size is represented by n1, the number of adjustment functions is represented by n2, and the predicted number of functions to be completed is represented by n2.

[0010] In some embodiments, there are multiple target software programs. The R&D evaluation result is determined based on multiple initial indicator scores and target R&D coefficients, including: obtaining the software R&D coefficient for each target software program, where the software R&D coefficient reflects the business risk level of the target software; for each target software program, determining the R&D indicator score corresponding to each target evaluation indicator based on the multiple initial indicator scores and the software R&D coefficients, thereby determining multiple R&D indicator scores for each target software program; for each target software program, determining the software efficiency score based on the multiple R&D indicator scores, thereby determining multiple software efficiency scores; and finally, determining the R&D evaluation result based on the multiple software efficiency scores and the target R&D coefficients.

[0011] In some embodiments, determining the software efficiency score of the target software based on multiple R&D indicator scores includes: determining whether all R&D indicator scores are non-zero. If all R&D indicator scores are non-zero, the multiple R&D indicator scores are added together to obtain the software efficiency score of the target software. If any one of the multiple R&D indicator scores is non-zero, the software efficiency score of the target software is determined based on the multiple R&D indicator scores and a preset indicator score threshold corresponding to each target evaluation indicator.

[0012] In some embodiments, determining the software efficiency score of the target software based on multiple R&D indicator scores and a preset indicator score threshold corresponding to each target evaluation indicator includes: determining a total R&D indicator score based on multiple R&D indicator scores; determining an actual total score based on a preset indicator score threshold corresponding to at least one target evaluation indicator whose R&D indicator score is not 0; summing the preset indicator score thresholds corresponding to multiple target evaluation indicators to obtain a preset total score; and determining the software efficiency score based on the total R&D indicator score, the actual total score, and the preset total score.

[0013] In some embodiments, the software efficiency score satisfies the following formula: .

[0014] in, Used to represent R&D indicator scores n is used to represent the total score of R&D indicators, and n3 is used to represent the number of target evaluation indicators. Used to represent the preset indicator scoring threshold corresponding to the target evaluation indicator. The value of m represents the total actual score, and m represents the number of target evaluation indicators whose R&D indicator scores are not zero. Used to represent the preset total score.

[0015] Secondly, this application provides an evaluation device for R&D work efficiency, which includes a response module, an acquisition module, and a determination module.

[0016] The response module is used to respond to evaluation instructions. The acquisition module is used to acquire multiple target evaluation metrics and target development coefficients. These metrics assess the efficiency of the target development team in completing development tasks, including deleting, modifying, and / or adding target functions to the target software. The target evaluation metrics include: target code coverage, unit test case effectiveness, and technical debt repair rate. The target code is used to implement the target functions. Target coverage indicates the extent to which unit test cases cover the target code. The technical debt repair rate is the ratio between the number of repaired technical debts and the previous technical debt. The number of repaired technical debts is the difference between the previous technical debt and the current technical debt. The previous technical debt refers to issues detected by static code testing of historical code, while the current technical debt refers to issues detected by static code testing of the target code. The target code is the updated historical code. The target development coefficient reflects the complexity of the development task. The determination module determines the initial score for each target evaluation metric based on the multiple target evaluation metrics. The determination module is also used to determine the R&D evaluation results based on multiple initial indicator scores and target R&D coefficients. The R&D evaluation results are used to indicate the efficiency of the target R&D team in completing R&D tasks.

[0017] Thirdly, this application provides an evaluation device for R&D work efficiency, the device comprising: a processor and a memory; the processor and the memory being coupled; the memory being used to store one or more programs, the one or more programs including computer-executable instructions, wherein when the R&D work efficiency evaluation device is running, the processor executes the computer-executable instructions stored in the memory to implement the method as described in the first aspect and any possible implementation thereof.

[0018] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect and any possible implementation thereof.

[0019] Fifthly, this application provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods described in the first aspect and any possible implementation thereof.

[0020] The technical problems that the evaluation device for R&D work efficiency, computer storage medium, or computer program product can solve and the technical effects that can be achieved in the above scheme can be found in the technical problems and technical effects solved in the first aspect above, and will not be repeated here. Attached Figure Description

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a method for evaluating the efficiency of research and development work, provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of a research and development work efficiency evaluation device provided in an embodiment of this application; Figure 3 A schematic diagram of another R&D work efficiency evaluation device provided in an embodiment of this application; Figure 4 A conceptual partial view of a computer program product provided for an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] In the description of this application, it should be understood that the terms "upper," "lower," "left," "right," "front," "rear," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or relative positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and for simplification, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Unless otherwise specified, the above-mentioned orientational descriptions can be flexibly set in practical applications, provided that the relative positional relationships shown in the accompanying drawings are satisfied.

[0024] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0025] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, article, or apparatus that includes that element.

[0026] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0027] In software development, low efficiency in R&D work can affect the development schedule, thus impacting the software's release date and ultimately the user experience. Therefore, how to evaluate the efficiency of R&D work has become a pressing technical problem to be solved.

[0028] In the field of team R&D quality assessment, traditional methods include: multi-team collaborative measurement models, contribution measurement algorithms, and automated auditing methods for process standardization. Among these, multi-team collaborative measurement models focus on multi-team task coupling models, team quality contribution algorithms, and cross-team pipeline maturity assessments. From a technical coverage perspective, this method does not cover non-code collaboration technologies (such as design reviews), soft indicators like knowledge transfer, or collaborative assessments during the requirements phase, resulting in lower accuracy. Contribution measurement algorithms are applied to digital fingerprints of the R&D process and team practice matching engines. In terms of effectiveness, this method cannot achieve 100% coverage in process compliance checks and cannot effectively reduce process fraud rates, leading to lower accuracy. Automated auditing methods for process standardization are mainly responsible for multi-team architecture alignment detection and version compatibility prediction. In terms of applicable scenarios, this method is biased towards cross-team collaboration in microservices and team-based iterations of large systems, resulting in incomplete assessments and lower accuracy.

[0029] Therefore, how to accurately assess the efficiency of R&D work has become a pressing technical problem that needs to be solved.

[0030] To address the aforementioned technical problems, this application provides a method for evaluating R&D work efficiency. In this method, in response to an evaluation instruction, the target unit test coverage, unit test case validity, and technical debt repair rate of the target code corresponding to the target software are obtained, and corresponding initial indicator scores are determined. This allows for the determination of the coverage of unit test cases to the target code, the validity of unit test cases, and the status of technical debt repair, thereby accurately determining the completion status of the target R&D team for R&D tasks such as deleting, modifying, and / or adding target functions to the target software, and thus determining the work efficiency of the target R&D team. Simultaneously, by obtaining the target R&D coefficient, the complexity of the R&D tasks can be considered, thereby accurately determining the R&D evaluation results and achieving an accurate assessment of the R&D work efficiency of the target R&D team.

[0031] The R&D efficiency evaluation method provided in this application can be applied to an R&D efficiency evaluation device, specifically to the processor of that device. The R&D efficiency evaluation device can be an electronic device such as a personal computer (PC), laptop computer, mobile device, tablet computer, or laptop computer. This application does not limit the specific form of the R&D efficiency evaluation device. Alternatively, the R&D efficiency evaluation device can also be a server, or a server cluster consisting of multiple servers. In some implementations, the server cluster can be a distributed cluster server. This application does not impose any limitations in this regard.

[0032] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0033] like Figure 1 As shown in the figure, a method for evaluating research and development work efficiency is provided in an embodiment of this application. The method includes: S101. In response to the evaluation instructions, obtain multiple target evaluation indicators and target R&D coefficients.

[0034] Among them, the evaluation instruction is the instruction that triggers the evaluation of the R&D team's work efficiency. It is used to clarify the user's evaluation needs for the R&D team's R&D work and initiate the calculation process between the target evaluation indicators and the target R&D coefficient.

[0035] Multiple target evaluation metrics are used to assess the efficiency of the target R&D team in completing R&D tasks. The target R&D team consists of multiple R&D members who can execute R&D tasks by writing target code. The target code is used to implement the target functions of the target software. The R&D tasks include: deleting, modifying and / or adding target functions of the target software.

[0036] Several target evaluation metrics include: target code coverage, unit test case effectiveness (also understood as unit test case efficiency), and technical debt repair rate. Target coverage indicates the extent to which unit test cases cover the target code; unit test cases are used to perform unit tests on the target code. Unit test case effectiveness is the ratio between effective unit test cases and the total number of unit test cases. Effective unit test cases are those that can continuously and reliably verify whether the behavior of code units (usually functions / methods) under specific conditions meets expectations. Technical debt repair rate is the ratio between the number of repaired technical debts and the number of previous technical debts. The number of repaired technical debts is the difference between the number of previous technical debts and the number of current technical debts. Previous technical debts were issues detected by static code testing of historical code, while current technical debts are issues detected by static code testing of the target code, which is the updated historical code. The number of technical debts is the number of current technical debts.

[0037] In some embodiments, the multiple target evaluation metrics may further include at least one of the following: requirement pass rate, design functional requirement pass rate, number of technical debts, static test pass rate of target code, automated test case development execution rate of target software, software submission test pass rate of target software, on-time submission functional test rate of target software, performance test pass rate of target software, number of performance inspection tests, number of online faults, number of online defects, number of unfixed issues and / or average duration of unfixed issues.

[0038] The requirement pass rate is the ratio of the number of qualified functional requirements to the total number of functional requirements. Qualified functional requirements are those that have passed the functional requirement review. The design functional requirement pass rate is the ratio of the number of qualified design functional requirements to the total number of functional requirements. Qualified design functional requirements are those corresponding to functional design schemes that have passed the design scheme review. The static test pass rate is the ratio of the number of static test passes within a preset time period to the total number of static tests. The number of static test passes is the number of times the target code meets the static test conditions when static testing is performed on the target code within the preset time period. Static testing is used to indicate testing the target code using static code scanning tools without running the target code. The automated test case development execution rate is the ratio of the number of target functional test cases run to the total number of functional test cases. Target functional test cases are the functional test cases run by the target R&D team, and functional test cases are used to test the target function. The software submission test pass rate is the ratio of the number of software submission test passes to the preset number of software submission tests. The number of software submission test passes is the number of times the target software has passed software submission tests. Software submission tests refer to the number of times a team member in the target R&D team submits the target software to a test object, allowing the test object to perform software testing on the target software. The on-time submission rate for functional tests is the ratio of the number of on-time functional submissions to the total number of functional submissions. The on-time submission rate is the number of times the target software launched within a preset time period is submitted for testing before the preset submission time. The total submission rate is the number of times the target software launched within the preset time period is submitted for testing. The performance test pass rate is the ratio of the performance test difference to a preset performance threshold. The performance test difference is the difference between the performance test data and the preset performance threshold, reflecting the magnitude of change in the performance test data. The performance test data is the test data obtained by performing performance tests on all functions of the target software, including at least one of the following: response time, throughput, concurrent users, and / or resource utilization. The number of performance inspection tests is the number of times the team performs automated performance tests on certain functions of the target software. The number of online failures is the number of software issues discovered after the target software is released. The number of online defects is the number of software issues discovered after the target software is submitted for testing but before its release. The number of unfixed issues is the number of software-related issues identified during the development phase but not fixed. The average duration of unresolved issues is the ratio of the sum of the durations of unresolved issues to the number of unresolved issues. The sum of the durations of all unresolved issues is the sum of the cumulative durations of all unresolved issues. The cumulative duration of unresolved issues is the time interval between the first and second time points, where the first time point is the moment when the unresolved issues were identified, and the second time point is the current time. The target R&D coefficient is used to reflect the complexity of the R&D task.

[0039] This application does not limit the method of triggering evaluation instructions. For example, the R&D work efficiency evaluation device is equipped with an evaluation touch button, and the user triggers the evaluation instruction by touching the evaluation touch button. Another example is that the R&D work efficiency evaluation device automatically generates and triggers the evaluation instruction at a preset evaluation time. This application does not limit the preset evaluation time. For example, the preset evaluation time is the last day of each month and also the first day of each week.

[0040] As one possible implementation, in response to evaluation instructions, the APIs of the test plan management system, CI / CD tools, test defect management system, access control system, automated test management system, and / or fault tracing system can be directly invoked to obtain multiple target evaluation metrics.

[0041] The following section details how to directly call the APIs of test plan management systems, CI / CD tools, test defect management systems, access control systems, automated test management systems, or fault tracing systems to obtain multiple target evaluation metrics.

[0042] (1) Demand pass rate One possible implementation involves calling the application programming interface (API) of the test plan management system to obtain the review data stored in the test plan management system; calling the test plan management API of the test plan management system to obtain the test plan table stored in the test plan management system; determining the total number of functional requirements based on the test plan table; determining the number of qualified functional requirements based on the test plan table and the review data; and determining the requirement pass rate based on the number of qualified functional requirements and the total number of functional requirements.

[0043] The test plan management system is an information system used for the full lifecycle management of test plans, enabling operations such as test plan creation, editing, publishing, querying, progress tracking, and resource allocation. The test plan table includes: the predicted development cycle, test submission time, and predicted deployment time for each functional requirement. When there are multiple test plans, the total number of functional requirements is the sum of the number of functional requirements across all test plan tables. Review data includes: the functional review results of the functional requirements, indicating whether a functional requirement has passed the functional review.

[0044] This application does not limit the review process of functional requirement reviews. For example, the review result can be obtained by reviewing the functional requirements through a machine model, or by reviewers reviewing the functional requirements. This application does not limit the number of reviewers. For example, in the case where reviewers review the functional requirements, the number of reviewers is greater than or equal to 2, and the reviewers need to include at least one tester from the test team.

[0045] Exemplarily, the requirement passing rate satisfies Formula One.

[0046] Formula One.

[0047] In some embodiments, regression requirements and joint debugging requirements do not belong to functional requirements. Regression requirements are the retest requirements for historical functions, that is, after fixing the problems of historical functions or making minor optimizations to historical functions, it is necessary to retest whether the historical functions are normal and whether new problems are introduced. Functional requirements are the test requirements for the docking of multiple modules or systems, that is, the requirements for testing whether the docking between multiple modules or systems is smooth, whether the data transmission is accurate, and whether the interaction is normal.

[0048] (2) Design passing rate As a possible implementation, call the requirement function scheduling requirement review API of the test plan management system to obtain the review data stored in the test plan management system; call the test plan management API of the test plan management system to obtain the test plan table stored in the test plan management system; based on the test plan table, determine the total number of functional requirements; based on the test plan table and the review data, determine the number of qualified design functional requirements; based on the number of qualified design functional requirements and the total number of functional requirements, determine the requirement passing rate.

[0049] Among them, the review data also includes: the design review result of the functional requirement, and the design review result is used to indicate whether the design scheme corresponding to the functional requirement passes the design scheme review. When conducting the design scheme review, relevant attachment documents need to be provided. For the introduction of the design scheme review, reference can be made to the introduction of the functional review, and this application will not elaborate on it.

[0050] Exemplarily, the design passing rate satisfies Formula Two.

[0051] Formula Two.

[0052] Among them, when calculating the design passing rate, functional requirements that do not require a design scheme can be excluded.

[0053] (3) Target coverage rate of the target code As one possible implementation, the API of a continuous integration (CI) or continuous delivery (CD) tool is invoked to obtain the logs of the CI / CD pipeline and / or the scan results of the unit test coverage plugin; based on the logs of the CI / CD pipeline and / or the scan results of the unit test coverage plugin, the unit test coverage of the target code is determined.

[0054] Among them, CI / CD tools can call the unit test coverage plugin to detect unit test cases. The logs of the CI / CD pipeline include: the unit test coverage of the target code. The unit test coverage plugin is used to detect the degree of coverage of the target code by the unit test cases. The scan results of the unit test coverage plugin include: the unit test coverage results of each line of the target code. The unit test coverage results are used to indicate whether the line of code is covered by the test.

[0055] For example, the target coverage rate of the target code can be the average of the unit test coverage rates of the target code that are not zero over a preset time period. For instance, if the CI / CD pipeline logs include: unit test coverage rate of the target code on January 16: 85%, unit test coverage rate of the target code on January 17: 0%, and unit test coverage rate of the target code on January 18: 88%, then the target coverage rate of the target code is determined to be 86.5%.

[0056] (4) Validity of unit test cases As one possible implementation, the API of the CI / CD tool is called to obtain the logs of the CI / CD pipeline and / or the scan results of the unit test validity plugin; based on the logs of the CI / CD pipeline and / or the scan results of the unit test validity plugin, the validity of the unit test code of the target code is determined.

[0057] Among them, CI / CD tools can call the unit test validity plugin to check unit test cases. The logs of the CI / CD pipeline include: the validity of unit test cases. The unit test validity plugin is used to check whether the unit test cases can continuously and reliably verify whether the behavior of code units (usually functions / methods) under specific conditions meets the expected unit tests. The scan results of the unit test coverage plugin include: the validity results of unit test cases. The validity results are used to indicate whether the unit test cases can continuously and reliably verify whether the behavior of code units (usually functions / methods) under specific conditions meets the expected unit tests.

[0058] (5) Technology debt recovery rate As one possible implementation, the API of the CI / CD tool is called to obtain the logs and / or access control data of the CI / CD pipeline; based on the logs and / or access control data of the CI / CD pipeline, the previous technical debt and the current technical debt are determined; based on the amount of the previous technical debt and the amount of the current technical debt, the technical debt repair rate of the target code is determined.

[0059] The CI / CD tool can invoke code scanning tools to perform static testing on the target code and obtain the static test results. Furthermore, the CI / CD tool can access historical test data from the access control system to obtain the historical data required for calculating access control verification results. The CI / CD pipeline logs include: static code scanning results and access control verification results. The static code testing results include the number of technical debts in the previous test and the number of technical debts in the current test. The access control verification results indicate whether the technical debts meet the preset technical debt passing conditions, which are that the number of technical debts is less than or equal to a preset technical debt threshold. The access control data includes: the number of technical debts in the current test, and the verification results showing the ratio of the difference between the current number of technical debts and the previous number of technical debts to the previous number of technical debts.

[0060] The previous technical debt includes the number of static test code issues and component issues in the current test, and also includes the number of historical static test code issues and problematic components. Technical debt, also known as technical debt issues, specifically refers to code-related problems written by developers during code development due to reasons such as rushing deadlines, simplifying implementations, temporarily avoiding problems, or compromising on technology choices. These issues may function normally in the short term without obvious functional failures, but pose long-term risks and significantly increase subsequent maintenance / iteration costs. Technical debt can be used to evaluate code quality.

[0061] For example, the number of code issues and issue components in the previous test was the number of code issues obtained by querying the CI / CD tool's API on the last day of the previous month. The number of code issues in this test is the number of code issues and issue components obtained by querying the CI / CD tool's API on the last day of this month. The preset code quantity threshold is 10. The static code scanning tool detects historical code and obtains the static code scanning results: the total number of historical code technical debt issues is 12. The access control data shows: the total number of historical code technical debt issues is 12, which fails the verification. The code scanning tool detects the target code and obtains the code scanning results: the total number of target code technical debt issues is 8. The access control data shows: the total number of target code technical debt issues is 8, which passes the verification. The technical debt repair rate of the target code is 0.33.

[0062] For example, the technology debt repair rate satisfies Formula 3.

[0063] Formula 3.

[0064] (6) Target code access control pass rate As one possible implementation, the API of the CI / CD tool is called to obtain CI / CD pipeline logs and / or access control data; based on the CI / CD pipeline logs and / or access control data, the number of static test passes and the total number of static tests within a preset time period are determined; based on the number of static test passes and the total number of static tests, the static test pass rate of the target code is determined.

[0065] The number of static test passes refers to the number of times the code passes through the access control checkpoints, and the total number of static tests refers to the total number of access control checkpoints. Thus, CI / CD pipeline logs and / or access control data can be used to determine whether the submitted code corresponding to the requirements meets the quality baseline, including whether the requirement quality, design quality, and static code scanning metrics meet the thresholds for initiating testing tasks. These thresholds are referred to as access control checkpoints.

[0066] For example, the pass rate of the target code access control satisfies Formula 4.

[0067] Formula 4.

[0068] In this way, the code gate pass rate can be used to determine whether the compiled code for implementing the target functions of the target software (deleting, modifying, and / or adding) meets the quality requirements for entering the test task.

[0069] (7) Execution rate of automated use case development for the target software One possible implementation involves calling the APIs of the automated test management system, the CI / CD tool, and the test plan management system to obtain automated functional test case data, CI / CD pipeline logs, and test plan tables. Based on the automated functional test case data, CI / CD pipeline logs, and test plan tables, the number of first and second automated functional test cases is determined. Based on the number of first and second automated functional test cases, the number of target automated functional test cases executed and the total number of automated functional test cases are determined. Based on the number of target automated functional test cases executed and the total number of automated functional test cases, the execution rate of automated test case development by the developers of the target software is determined.

[0070] The automated functional test case data includes: automated functional test cases. The CI / CD pipeline log includes: the creation object corresponding to the automated functional test case, and the execution identifier of the automated functional test case. The execution identifier indicates whether the automated functional test case has been run by the creation object. The first automated functional test case is the automated functional test case created and run by the target R&D team within the test plan table, i.e., the target automated functional test case. The second automated functional test case is the automated functional test case created and run by the test object within the test plan table. The total number of automated functional test cases is the sum of the number of first automated functional test cases and the number of second automated functional test cases. Deduplication is required when determining the number of automated functional test cases.

[0071] The objects to be created include development objects and test objects. For example, the test object can be a developer, and the objects to be created for automated functional test cases are developers or testers.

[0072] For example, the execution rate of automated use case development for the target software satisfies Formula 5.

[0073] Formula 5.

[0074] For example, the automation use case development execution rate of the target R&D team satisfies Formula Six.

[0075] Formula Six.

[0076] Therefore, using the automated test case development execution rate as an evaluation metric allows developers to perform more automated tests before the test subjects test the target software, thereby discovering and fixing potential problems in advance, reducing the number of times the test submission process is initiated, and improving the efficiency of R&D work.

[0077] (8) Software submission test pass rate of the target software One possible implementation involves calling the API of the CI / CD tool to obtain CI / CD pipeline logs and / or access control data; calling the test plan management API of the test plan management system to obtain the test plan table; based on the test plan table, CI / CD pipeline logs and / or access control data, determining the test submission pass orders and the total number of test submission work orders; based on the test submission pass orders and the total number of test submission work orders, determining the number of times the software was submitted for testing and the total number of times the software was submitted for testing; and based on the number of times the software was submitted for testing and the total number of times the software was submitted for testing, determining the software submission test pass rate.

[0078] The CI / CD pipeline log includes: Test submission pass tickets, which are structured task documents initiated by the target development team to deliver target code or functionality to the testing team for software submission testing. Access control data includes: Test submission pass tickets. The test plan table includes: Total test submission tickets. The number of test submission pass tickets in the test plan table represents the number of times the software submission test has passed, and the total number of test submission tickets represents the total number of times the software has been submitted for testing. When there are multiple test plan tables, the number of times the software has been submitted for testing is the number of test submission pass tickets in all test plan tables, and the total number of test submission tickets is the sum of the number of test submission tickets in all test plan tables.

[0079] For example, the pass rate of the target software's software submission test satisfies Formula 7.

[0080] Formula 7.

[0081] In some embodiments, when calculating the initial score for software metrics, if there are no test requests for any of the deployed functionalities in a system, the initial score for that specific metric for the target software is 0. When calculating the initial score for team metrics, the test requests for the remaining functionalities of the target software need to be added when calculating the total number of software test submissions. However, the number of successful software test submissions is not counted in the number of successful test requests for the target software; that is, the number of successful test requests for the target software is 0, and the number of successful software test submissions is 0.

[0082] (9) On-time delivery rate of the target software's submission function test As one possible implementation, based on the test plan, CI / CD pipeline logs, and / or access control data, determine the on-time test submission work order and the total test submission work order; based on the on-time test submission work order and the total test submission work order, determine the number of on-time test function points and the total number of test function points; based on the number of on-time test function points and the total number of test function points, determine the on-time submission rate of the target software's functional testing.

[0083] The test plan includes the planned start date. If the test plan also includes a planned submission time, the default submission time is the planned submission time. If the test plan does not include a planned submission time, the default submission time is the planned start date.

[0084] For example, the on-time testing rate of the target software's function points satisfies Formula 8.

[0085] Formula 8.

[0086] (10) Performance test pass rate and number of performance inspection tests of the target software As one possible implementation, the test plan management API of the test plan management system and the performance test plan management API of the automated test management system are called to obtain the performance test results, preset performance thresholds, and number of performance inspection tests of the target software within the test plan; based on the performance test results and preset performance thresholds of the target software, the performance test pass rate is determined.

[0087] For example, the pass rate of the target software's performance test satisfies Formula Nine.

[0088] Formula Nine.

[0089] In some embodiments, the number of performance inspection tests is the total number of times performance tests are performed.

[0090] (11) Number of online faults Online faults include: self-reported non-originating faults, non-self-reported non-originating faults, self-reported originating faults, and non-self-reported originating faults. Self-reported non-originating faults are software faults discovered by the target software's development team that are not caused by the target software. Non-self-reported non-originating faults are software faults discovered by external parties that are not caused by the target software. Self-reported originating faults are software faults discovered by the target software's development team that are inherent in the target software itself. Non-self-reported originating faults are software faults discovered by external parties that are inherent in the target software itself. External parties can be the target software's development team (not belonging to the target software), quality management departments, software users, or social media platforms. For example, software faults discovered based on user complaint data are classified as non-self-reported non-originating faults or non-self-reported originating faults.

[0091] One possible implementation is to call the APIs of the fault tracing system and the automated test management system to obtain software problem data; based on the software problem data, determine the online fault data; and based on the online fault data, determine the number of online faults.

[0092] The number of online faults that users are unaware of and have already fixed can be excluded from the statistics.

[0093] (12) Number of online defects The number of online defects includes: pre-production testing defects, production verification defects, and post-deployment defects. Pre-production testing defects are software issues identified during the system testing phase (i.e., the phase where the target software is submitted for testing). Production verification defects are software issues identified on the day of deployment during smoke testing, release verification, or business walkthroughs in the production environment. Post-deployment defects are software issues discovered after deployment through production walkthroughs, monitoring alerts, user feedback, and other means. As one possible implementation, the APIs of the fault tracing system and the automated test management system are called to obtain software problem data; based on the software problem data, the number of pre-production test defects, the number of pre-production test defects, and the number of defects after deployment are determined; based on the number of pre-production test defects, the number of pre-production test defects, and the number of defects after deployment, the number of online defects is determined.

[0094] The software issue data includes: the number of defects in pre-production testing, the number of defects in pre-production testing, and the number of defects after deployment. (13) Average duration of unresolved issues and number of unresolved issues As one possible implementation, the APIs of the test defect management system and the automated test management system are called to obtain software problem data; based on the software problem data, the duration of multiple unfixed problems and the number of unfixed problems are determined; based on the duration of multiple unfixed problems, the sum of the durations of unfixed problems is determined; based on the number of unfixed problems, the number of unfixed problems is determined; based on the sum of the durations of unfixed problems and the number of unfixed problems, the average duration of unfixed problems is determined.

[0095] The software issue data includes: unresolved issues and duration of unresolved issues. The duration of unresolved issues is the time period between the time the unresolved issue is identified and the preset evaluation time.

[0096] The number of unresolved issues reflects the system's defect response speed during the query cycle, which can accurately assess software quality and predict system stability and risks.

[0097] In one possible design, the average duration of unresolved issues satisfies Formula 10.

[0098] Formula 10.

[0099] As another possible implementation, before executing S101, the APIs of the test plan management system, CI / CD tool, test defect management system, access control system, automated test management system, and fault tracing system can be called to obtain raw indicator data, which is then stored in the quality database. In response to the evaluation instruction, initial indicator data is retrieved from the quality database, and multiple target evaluation indicators are determined based on this initial indicator data.

[0100] The following sections, S1011-S1014, explain how to obtain the target R&D coefficient.

[0101] S1011. Determine the initial R&D coefficients corresponding to the target R&D team.

[0102] The initial R&D coefficient is related to the amount of code in the target software provided by the target R&D team, as well as the number of target functions corresponding to the completion of the R&D task.

[0103] As one possible implementation, the number of completed target functions, the first average code volume, the second average code volume, the number of adjusted functions, and the predicted number of completed functions are obtained; based on the number of completed target functions, the first average code volume, the second average code volume, the number of adjusted functions, and the predicted number of completed functions, the initial R&D coefficient is determined.

[0104] The first average code volume refers to the average code volume of the target R&D team, i.e., the average code volume of each team object within the target R&D team. The first average code volume is the ratio between the total team code volume and the number of team objects. The total team code volume is the sum of the code volumes of all team objects within the target R&D team, and the number of team objects is the total number of team objects within the target R&D team. The code volume can be either lines of code or functions; this application does not impose any restrictions on this. The second average code volume refers to the average code volume of the target R&D department to which the target R&D team belongs, i.e., the average code volume of each department object within the target R&D department. The second average code volume is the ratio between the total department code volume and the number of department objects. The total department code volume is the sum of the code volumes of all target R&D teams within the target R&D department, and the number of department objects is the total number of department objects within the target R&D department. The number of completed target functions includes the number of target functions that require deletion, addition, and modification of code. The number of adjusted functions refers to the number of target functions that the target R&D team needs to add and / or modify code for. The predicted number of functions is the number of target functions that the target R&D team expects to complete, referring to the total number of target functions that the target R&D team is responsible for. The predicted number of features includes: the number of target features that have been completed and the number of target features that have not been completed.

[0105] For example, the team object could be a developer.

[0106] The amount of code and the number of target features can reflect the workload of the target team. By calculating the number of completed target features, the first average code volume, the second average code volume, the number of adjusted features, and the predicted number of completed features, the initial development coefficient corresponding to the target R&D team can be accurately determined.

[0107] Specifically, the initial R&D coefficient satisfies Formula 11.

[0108] Formula 11.

[0109] in, The initial R&D coefficient is represented by N1, the number of target functions completed is represented by Lavg1, the first average code size is represented by Lavg2, the second average code size is represented by n1, the number of adjustment functions is represented by n2, and the predicted number of functions to be completed is represented by n2.

[0110] The number of completed target features reflects the achievements of the target R&D team; the first average code size reflects the R&D team's R&D capabilities; the second average code size serves as a benchmark for the department's R&D capabilities; the adjusted number of features reflects the effective work tasks of the target R&D team; and the predicted number of completed features reflects all the work tasks the target R&D team needs to complete. This is determined by the code ratio between the first and second average code sizes (i.e.,...). This method accurately quantifies the contribution of the target development team to the overall department's work and precisely determines the target team's code output efficiency within the department. By multiplying the number of completed target features by the code output ratio, it reflects the capability differences between the target development team and other development teams. For example, for teams completing the same 10 features, those with higher capabilities than the department average will have a higher initial development coefficient, while teams with lower capabilities will have a proportionally lower initial development coefficient. The percentage of workload to be adjusted (i.e.,...) The initial R&D coefficient reflects the difficulty of the target R&D task. A larger proportion of the workload to be adjusted indicates a greater workload and higher difficulty in the target R&D task. Therefore, multiplying the number of completed target functions by the code ratio, and then multiplying that by the proportion of the workload to be adjusted, can correct the initial R&D coefficient. This allows the initial R&D coefficient calculation result to more realistically, comprehensively, and dynamically reflect the current situation of the R&D team, thereby accurately reflecting the target R&D team's work capabilities and, consequently, more accurately assessing the target R&D team's work efficiency.

[0111] S1012. Monotonically sort the initial R&D coefficients corresponding to multiple target R&D teams.

[0112] The initial R&D coefficients corresponding to multiple target R&D teams can be monotonically sorted in either ascending order or descending order. This application does not impose any restrictions on this.

[0113] For example, the initial R&D coefficients for the 12 target R&D teams are as follows: initial R&D coefficient 1 is 0.00, initial R&D coefficient 2 is 1.55, initial R&D coefficient 3 is 22.63, initial R&D coefficient 4 is 35.98, initial R&D coefficient 5 is 24.01, initial R&D coefficient 6 is 44.86, initial R&D coefficient 7 is 29.76, initial R&D coefficient 8 is 33.03, initial R&D coefficient 9 is 110.81, initial R&D coefficient 10 is 0.02, initial R&D coefficient 11 is 197.62, and initial R&D coefficient 12 is 203.33. The initial R&D coefficients of the 12 target R&D teams are monotonically sorted in ascending order, resulting in the following 12 initial R&D coefficients: 0.00, 0.02, 1.55, 22.63, 24.01, 29.76, 33.03, 35.98, 44.86, 110.81, 197.62, and 203.33.

[0114] S1013. Based on the sorted initial R&D coefficients, group them to obtain a set of monotonically sorted initial R&D coefficients.

[0115] For example, the coefficients of the 12 initial R&D coefficients after sorting are divided into 4 groups, resulting in 4 monotonically sorted sets of initial R&D coefficients, each containing 3 initial R&D coefficients. The 4 monotonically sorted sets of initial R&D coefficients are: Initial R&D Coefficient Set 1, Initial R&D Coefficient Set 2, Initial R&D Coefficient Set 3, and Initial R&D Coefficient Set 4. Specifically, Initial R&D Coefficient Set 1 includes: 0.00, 0.02, and 1.55; Initial R&D Coefficient Set 2 includes: 22.63, 24.01, and 29.76; Initial R&D Coefficient Set 3 includes: 33.03, 35.98, and 44.86; and Initial R&D Coefficient Set 4 includes: 110.81, 197.62, and 203.33.

[0116] S1014. Based on the sequence number of each initial R&D coefficient set, determine the target R&D coefficient corresponding to the initial R&D coefficient set.

[0117] In one possible implementation, the device for evaluating the efficiency of R&D work pre-stores a coefficient correspondence, which is a correspondence between a preset set number and a preset R&D coefficient; based on the number of each initial R&D coefficient set and the coefficient correspondence, the target R&D coefficient corresponding to the initial R&D coefficient set is determined.

[0118] The preset set number includes the number of each initial R&D coefficient set, and the preset R&D coefficients include the target R&D coefficients corresponding to the initial R&D coefficient sets.

[0119] For example, the coefficient correspondence includes: preset R&D coefficient 1 corresponding to preset number 1 is 0.8, preset R&D coefficient 2 corresponding to preset number 2 is 0.0, preset R&D coefficient 3 corresponding to preset number 3 is 1.0, and preset R&D coefficient 4 corresponding to preset number 4 is 1.1. Therefore, the target R&D coefficient corresponding to initial R&D coefficient set 1 is 0.8, the target R&D coefficient corresponding to initial R&D coefficient set 2 is 0.9, the target R&D coefficient corresponding to initial R&D coefficient set 3 is 1.0, and the target R&D coefficient corresponding to initial R&D coefficient set 4 is 1.1. In this way, the target R&D coefficient corresponding to each target R&D team can be determined. For example, the target R&D coefficient corresponding to target R&D team 1 is 0.8, and the target R&D coefficient corresponding to initial R&D coefficient 3 is 0.9.

[0120] As can be seen from S1011-S1014, since the initial R&D coefficient is related to the amount of code of the target software provided by the target R&D team and the number of target functions corresponding to the completion of the R&D task, by monotonically sorting and grouping multiple initial R&D coefficients and determining the corresponding target R&D coefficient for each set of initial R&D coefficients, the workload of multiple target R&D teams can be determined, thereby making the obtained target R&D coefficient more accurate and able to be accurately used to evaluate the work efficiency of the target R&D team.

[0121] S102. Based on multiple target evaluation indicators, determine the initial indicator score corresponding to each target evaluation indicator among the multiple target evaluation indicators.

[0122] Among them, the requirements pass rate, design functional requirements pass rate, target code target coverage rate, unit test case effectiveness, technical debt repair rate, target code static test pass rate, target software automated test case development execution rate, target software software submission test pass rate, target software submission functional test on-time rate, and target software performance test pass rate are positively correlated with their corresponding initial indicator scores. The number of static code issues, the number of online failures, the number of online defects, the number of unfixed issues, and / or the average duration of unfixed issues are negatively correlated with their corresponding initial indicator scores. Each initial indicator score corresponds to at least one target evaluation indicator.

[0123] For example, a requirement pass rate corresponds to an initial indicator score of 1. If the requirement pass rate is less than 0.8, the initial indicator score 1 is 0; if the requirement pass rate is greater than or equal to 0.8 and less than 0.9, the initial indicator score 1 is 1; if the requirement pass rate is greater than or equal to 0.9 and less than 0.93, the initial indicator score 1 is 2; if the requirement pass rate is greater than or equal to 0.93 and less than 0.96, the initial indicator score 1 is 3; if the requirement pass rate is greater than or equal to 0.96 and less than 1, the initial indicator score 1 is 4; and if the requirement pass rate is equal to 1, the initial indicator score 1 is 5.

[0124] For example, a design functional requirement pass rate corresponds to an initial indicator score of 2. If the design functional requirement pass rate is less than 0.8, the initial indicator score 2 is 0; if the design functional requirement pass rate is greater than or equal to 0.8 and less than 0.9, the initial indicator score 2 is 1; if the design functional requirement pass rate is greater than or equal to 0.9 and less than 0.93, the initial indicator score 2 is 2; if the design functional requirement pass rate is greater than or equal to 0.93 and less than 0.96, the initial indicator score 2 is 3; if the design functional requirement pass rate is greater than or equal to 0.96 and less than 1, the initial indicator score 2 is 4; and if the design functional requirement pass rate is equal to 1, the initial indicator score 2 is 5.

[0125] For example, a target coverage rate corresponds to an initial metric score of 3. If the target coverage rate is less than 0.7, the initial indicator score 3 corresponding to the target coverage rate is 0; if the target coverage rate is greater than or equal to 0.7 and less than 0.75, the initial indicator score 3 corresponding to the target coverage rate is 1; if the target coverage rate is greater than or equal to 0.75 and less than 0.8, the initial indicator score 3 corresponding to the target coverage rate is 2; if the target coverage rate is greater than or equal to 0.8 and less than 0.85, the initial indicator score 3 corresponding to the target coverage rate is 3; if the target coverage rate is greater than or equal to 0.85 and less than 0.9, the initial indicator score 3 corresponding to the target coverage rate is 4; if the target coverage rate is greater than or equal to 0.9 and less than 0.95, the initial indicator score 3 corresponding to the target coverage rate is 5; if the target coverage rate is greater than or equal to 0.95 and less than 1, the initial indicator score 3 corresponding to the target coverage rate is 6; if the target coverage rate is equal to 1, the initial indicator score 3 corresponding to the target coverage rate is 7.

[0126] For example, the validity of a unit test case corresponds to an initial metric score of 4. If the validity of a unit test case is less than 0.7, the initial metric score 4 corresponding to the validity of the unit test case is 0; if the validity of a unit test case is greater than or equal to 0.7, and the validity of a unit test case is less than 0.75, the initial metric score 4 corresponding to the validity of the unit test case is 1; if the validity of a unit test case is greater than or equal to 0.75, and the validity of a unit test case is less than 0.8, the initial metric score 4 corresponding to the validity of the unit test case is 2; if the validity of a unit test case is greater than or equal to 0.8, and the validity of a unit test case is less than 0.85, the initial metric score 4 corresponding to the validity of the unit test case is 4. 3; If the validity of a unit test case is greater than or equal to 0.85 and less than 0.9, then the initial metric score for the validity of the unit test case is 4; if the validity of a unit test case is greater than or equal to 0.9 and less than 0.95, then the initial metric score for the validity of the unit test case is 5; if the validity of a unit test case is greater than or equal to 0.95 and less than 1, then the initial metric score for the validity of the unit test case is 6; if the validity of a unit test case is equal to 1, then the initial metric score for the validity of the unit test case is 7.

[0127] For example, an initial metric score of 5 corresponds to both the issue fix rate and the number of static code issues. If the issue fix rate is less than 0.1, or the number of static code issues is greater than 28, the initial metric score of 5 for the issue fix rate is 0; if the issue fix rate is greater than or equal to 0.1 and less than 0.2, or the number of static code issues is less than or equal to 28 and greater than 25, the initial metric score of 5 for the issue fix rate is 1; if the issue fix rate is greater than or equal to 0.2 and less than 0.3, or the number of static code issues is less than or equal to 25 and greater than 22, the initial metric score of 5 for the issue fix rate is 2; if the issue fix rate is greater than or equal to 0.3 and less than 0.4, or the number of static code issues is less than or equal to 22 and greater than 19, the initial metric score of 5 for the issue fix rate is 3; if the issue fix rate is greater than... If the static code bug score is 0.4 and the bug fix rate is less than 0.5, or the number of static code bugs is less than or equal to 19 and the number of static code bugs is greater than 16, then the initial score for the bug fix rate is 4. If the bug fix rate is greater than or equal to 0 and the bug fix rate is less than 0.6, or the number of static code bugs is less than or equal to 16 and the number of static code bugs is greater than 13, then the initial score for the bug fix rate is 5. If the bug fix rate is greater than or equal to 0.6 and the bug fix rate is less than 0.7, or the number of static code bugs is less than or equal to 13 and the number of static code bugs is greater than 10, then the initial score for the bug fix rate is 6. If the bug fix rate is greater than or equal to 0.7, or the number of static code bugs is less than or equal to 10 and the number of static code bugs is greater than 6, then the initial score for the bug fix rate is 7. If the number of static code issues is less than or equal to 6, and the number of static code issues is greater than or equal to 4, then the initial indicator score corresponding to the number of static code issues is 8; if the number of static code issues is less than or equal to 3, and the number of static code issues is greater than or equal to 1, then the initial indicator score corresponding to the number of static code issues is 9; if the number of static code issues is 0, then the initial indicator score corresponding to the number of static code issues is 10.

[0128] For example, a static test pass rate corresponds to an initial indicator score of 6. If the static test pass rate is less than 0.1, the initial indicator score 6 is 0; if the static test pass rate is greater than or equal to 0.1 and less than 0.2, the initial indicator score 6 is 1; if the static test pass rate is greater than or equal to 0.2 and less than 0.3, the initial indicator score 6 is 2; if the static test pass rate is greater than or equal to 0.3 and less than 0.4, the initial indicator score 6 is 3; if the static test pass rate is greater than or equal to 0.4 and less than 0.5, the initial indicator score 6 is 4; if the static test pass rate is greater than or equal to 0.5 and less than 0.5, the initial indicator score 6 is 6. If the pass rate is less than 0.6, the initial indicator score for the static test pass rate is 5; if the static test pass rate is greater than or equal to 0.6 and less than 0.7, the initial indicator score for the static test pass rate is 6; if the static test pass rate is greater than or equal to 0.7 and less than 0.8, the initial indicator score for the static test pass rate is 7; if the static test pass rate is greater than or equal to 0.8 and less than 0.9, the initial indicator score for the static test pass rate is 8; if the static test pass rate is greater than or equal to 0.9 and less than 1, the initial indicator score for the static test pass rate is 9; if the static test pass rate is equal to 1, the initial indicator score for the static test pass rate is 10.

[0129] For example, an automated test case development execution rate corresponds to an initial metric score of 7. If the automated test case development execution rate is less than 0.05, the initial metric score 7 is 0; if the automated test case development execution rate is greater than or equal to 0.05 and less than 0.15, the initial metric score 7 is 0.5; if the automated test case development execution rate is greater than or equal to 0.15 and less than 0.3, the initial metric score 7 is 1; if the automated test case development execution rate is greater than or equal to 0.3 and less than 0.45, the initial metric score 7 is 1.5; and if the automated test case development execution rate is greater than or equal to 0.45, the initial metric score 7 is 2.

[0130] For example, a software submission test pass rate corresponds to an initial metric score of 8. If the software submission test pass rate is less than 0.1, the initial metric score 8 is 0; if the software submission test pass rate is greater than or equal to 0.1 and less than 0.2, the initial metric score 8 is 1; if the software submission test pass rate is greater than or equal to 0.2 and less than 0.3, the initial metric score 8 is 2; if the software submission test pass rate is greater than or equal to 0.3 and less than 0.4, the initial metric score 8 is 3; if the software submission test pass rate is greater than or equal to 0.4 and less than 0.5, the initial metric score 8 is 4; if the software submission test pass rate is greater than or equal to 0.5 and less than 0.5, the initial metric score 8 is 5; if the software submission test pass rate is greater than or equal to 0.5 and ...5, the initial metric score 8 is 5; if the software submission test pass rate is greater than or equal to 0.5 and less than 0.5, the initial metric score 8 is If the pass rate is less than 0.6, the initial score for the software submission test pass rate is 5; if the software submission test pass rate is greater than or equal to 0.6 and less than 0.7, the initial score for the software submission test pass rate is 6; if the software submission test pass rate is greater than or equal to 0.7 and less than 0.8, the initial score for the software submission test pass rate is 7; if the software submission test pass rate is greater than or equal to 0.8 and less than 0.9, the initial score for the software submission test pass rate is 8; if the software submission test pass rate is greater than or equal to 0.9 and less than 1, the initial score for the software submission test pass rate is 9; if the software submission test pass rate is equal to 1, the initial score for the software submission test pass rate is 10.

[0131] For example, the on-time performance of a submitted feature test corresponds to an initial metric score of 9. If the on-time submission rate of a functional test is less than 0.5, the initial indicator score (9) corresponding to the on-time submission rate is 0; if the on-time submission rate is greater than or equal to 0.5 and less than 0.55, the initial indicator score (9) corresponding to the on-time submission rate is 1; if the on-time submission rate is greater than or equal to 0.55 and less than 0.6, the initial indicator score (9) corresponding to the on-time submission rate is 2; if the on-time submission rate is greater than or equal to 0.6 and less than 0.65, the initial indicator score (9) corresponding to the on-time submission rate is 3; if the on-time submission rate is greater than or equal to 0.65 and less than 0.7, the initial indicator score (9) corresponding to the on-time submission rate is 4; and if the on-time submission rate is greater than or equal to 0.7, the initial indicator score (9) corresponding to the on-time submission rate is 5.

[0132] For example, a performance test pass rate and a performance inspection test count correspond to an initial metric score of 10. If the performance test pass rate is less than -0.3, the initial metric score 10 is 0; if the performance test pass rate is greater than or equal to -0.3 and less than -0.15, the initial metric score 10 is 6; if the performance test pass rate is greater than or equal to -0.15 and less than 0.3, the initial metric score 10 is 8; if the performance test pass rate is greater than or equal to 0.3, the initial metric score 10 is 10. If the number of performance inspection tests is greater than or equal to 1, and the performance test result indicates no test defects or that defects have been fixed, the initial metric score 10 corresponding to the performance test pass rate is increased by 2. For example, if the performance test pass rate is -0.2, the number of performance inspection tests is 1, and the performance test results are used to indicate no test defects, then the initial index score 10 corresponding to the performance test pass rate is 8; if the performance test pass rate is -0.2, the number of performance inspection tests is 1, and the performance test results are used to indicate test defects, then the initial index score 10 corresponding to the performance test pass rate is 6.

[0133] For example, one online fault corresponds to one initial metric score of 11. The maximum score for the initial metric score 11 corresponding to the number of online faults is 32 points. One point is deducted for each self-reported non-originating fault, four points for each non-self-reported non-originating fault, two points for each self-reported originating fault, and eight points for each non-self-reported originating fault. The lowest possible initial metric score 11 corresponding to the number of online faults is 0 points. If the online faults of target software 1 include: one self-reported non-originating fault, one non-self-reported non-originating fault, one self-reported originating fault, and one non-self-reported originating fault, then the initial metric score 11 corresponding to the number of online faults of target software 1 is 17.

[0134] For example, one online defect corresponds to an initial metric score of 12. The maximum score for the initial metric score of 12 for the number of online defects is 10 points, with 0.5 points deducted for each pre-production testing defect, 1 point deducted for each production verification defect, and 1 point deducted for each post-deployment walkthrough defect. The minimum initial metric score of 12 for the number of online defects is 0 points. If the online defects of target software 1 include: 1 pre-production testing defect, 1 production verification defect, and 1 post-deployment walkthrough defect, then the initial metric score of 12 for the number of online defects of target software 1 is 7.5.

[0135] For example, the number of unresolved issues and the average duration of unresolved issues correspond to an initial metric score of 13. If the number of unresolved issues is greater than 7, or the average duration of unresolved issues is greater than 28, then the initial metric score 13 corresponding to the number of unresolved issues is 0; if the number of unresolved issues is 7, or the average duration of unresolved issues is greater than 25 and less than or equal to 28, then the initial metric score 13 corresponding to the number of unresolved issues is 0.5; if the number of unresolved issues is 6, or the average duration of unresolved issues is greater than 21 and less than or equal to 25, then the initial metric score 13 corresponding to the number of unresolved issues is 1; if the number of unresolved issues is 5, or the average duration of unresolved issues is greater than 18 and less than or equal to 21, then the initial metric score 13 corresponding to the number of unresolved issues is 1.5; if the number of unresolved issues is 4, or the average duration of unresolved issues is greater than 14 and less than or equal to 28 ...0.5; if the number of unresolved issues is 6, or the average duration of unresolved issues is greater than 21 and less than or equal to 25, then the initial metric score 13 corresponding to the number of unresolved issues is 1; if the number of unresolved issues is 5, or the average duration of unresolved issues is greater than 18 and less than or equal to 21, then the initial If the average duration of unresolved issues is less than or equal to 18, the initial index score (13) corresponding to the number of unresolved issues is 2; if the number of unresolved issues is 3, or the average duration of unresolved issues is greater than 11 and less than or equal to 14, the initial index score (13) corresponding to the number of unresolved issues is 2.5; if the number of unresolved issues is 2, or the average duration of unresolved issues is greater than 7 and less than or equal to 11, the initial index score (13) corresponding to the number of unresolved issues is 3; if the number of unresolved issues is 1, or the average duration of unresolved issues is greater than 4 and less than or equal to 7, the initial index score (13) corresponding to the number of unresolved issues is 3.5; if the number of unresolved issues is 0, or the average duration of unresolved issues is less than or equal to 4, the initial index score (13) corresponding to the number of unresolved issues is 4.

[0136] If the initial indicator score corresponding to the average duration of unresolved issues is different from the initial indicator score corresponding to the number of unresolved issues, then the lowest score among the initial indicator scores corresponding to the average duration of unresolved issues and the number of unresolved issues is determined as the initial indicator score 13.

[0137] Optionally, the multiple target evaluation metrics include: bonus metrics and process metrics. If the initial metric score for any process metric is 0, then the initial metric score for the bonus metric is also 0; if the initial metric scores for all process metrics are not 0, then the initial metric score for the bonus metric is the actual score. For example, the automated test case development execution rate of the target software is a bonus metric, while the target code coverage, unit test case validity, and technical debt repair rate are process metrics. If the actual score for the initial metric score corresponding to the automated test case development execution rate is 1, the initial metric score corresponding to the target code coverage is 2, the initial metric score corresponding to the unit test case validity is 2, and the initial metric score corresponding to the technical debt repair rate is 2, then the initial metric score corresponding to the automated test case development execution rate is 1; if the actual score for the initial metric score corresponding to the automated test case development execution rate is 1, the initial metric score corresponding to the target code coverage is 2, the initial metric score corresponding to the unit test case validity is 2, and the initial metric score corresponding to the technical debt repair rate is 0, then the initial metric score corresponding to the automated test case development execution rate is 0.

[0138] S103. Determine the R&D evaluation results based on multiple initial indicator scores and target R&D coefficients.

[0139] The R&D evaluation results are used to indicate the efficiency of the target R&D team in completing R&D tasks.

[0140] In one possible implementation, the number of target software programs is multiple. S103 can be implemented as follows: S1031. Obtain the software development coefficient of each target software among multiple target software.

[0141] Among them, the software development coefficient is used to reflect the degree of business risk of the target software.

[0142] As one possible approach, the software development coefficient for each of the multiple target software programs is determined based on the software type of each target software.

[0143] Optionally, the software types include: customer service type, customer service management type, and employee management type. Customer service type software is used to instruct the provision of business services to end customers; customer service management type software is used to instruct the management of customer service software; and employee management type software is used by the enterprise to manage its internal employees. The software development coefficient corresponding to the customer service type is greater than that corresponding to the customer service management type, and the software development coefficient corresponding to the customer service management type is greater than that corresponding to the employee management type. For example, the software development coefficient corresponding to the customer service type is 1.1, the software development coefficient corresponding to the customer service management type is 1.0, and the software development coefficient corresponding to the employee management type is 0.9.

[0144] S1032. For each target software, based on multiple initial indicator scores and software development coefficients, determine the development indicator score corresponding to each target evaluation indicator, so as to determine the multiple development indicator scores corresponding to each target software.

[0145] For example, the software development coefficient of target software 1 is 1.1, the target coverage rate is 0.78, and the initial indicator score 3 corresponding to the target coverage rate is 2 points. Therefore, the development indicator score corresponding to the target coverage rate is 2.2 points.

[0146] S1033. For each target software, determine the software efficiency score based on multiple R&D indicator scores, thereby determining multiple software efficiency scores.

[0147] In one possible implementation, the software efficiency score of the target software is determined based on multiple R&D indicator scores, including: determining whether all R&D indicator scores are non-zero; if all R&D indicator scores are non-zero, then the multiple R&D indicator scores are added together to obtain the software efficiency score of the target software; if any one of the multiple R&D indicator scores is non-zero, then the software efficiency score of the target software is determined based on the multiple R&D indicator scores and the preset indicator score threshold corresponding to each target evaluation indicator.

[0148] For example, target software 1 is customer service software, target software 2 is customer service management software, and target software 3 is employee management software. Then, the software efficiency score of target software 1 satisfies Formula Thirteen.

[0149] Formula Thirteen.

[0150] in, Used to represent the initial indicator score. n3 is used to represent the R&D indicator score of target software 1, and n3 is used to represent the number of target evaluation indicators.

[0151] The software efficiency score of target software 2 satisfies Formula Fourteen.

[0152] Formula Fourteen.

[0153] in, Used to represent the R&D indicator score of target software 2.

[0154] The software efficiency score of target software 3 satisfies Formula 15.

[0155] Formula 15.

[0156] in, Used to represent the R&D indicator score of target software 3.

[0157] If multiple R&D indicator scores are all non-zero, the software efficiency score can be obtained by simply adding each R&D indicator score together. However, if any one of the target software's R&D indicator scores is non-zero, directly adding each R&D indicator score will result in a software efficiency score that differs significantly from the scores of other target software where all R&D indicator scores are non-zero, making it impossible to accurately assess the work efficiency of the target R&D team. Therefore, a software efficiency score can be calculated based on multiple R&D indicator scores and preset indicator score thresholds corresponding to each target evaluation indicator. This reduces the difference compared to target software where all R&D indicator scores are non-zero, thereby accurately assessing the work efficiency of the target R&D team.

[0158] One possible implementation involves determining the software efficiency score of the target software based on multiple R&D indicator scores and preset indicator scoring thresholds corresponding to each target evaluation indicator. This includes: determining the total R&D indicator score based on multiple R&D indicator scores; determining the actual total score based on the preset indicator scoring thresholds corresponding to at least one target evaluation indicator with a non-zero R&D indicator score; summing the preset indicator scoring thresholds corresponding to multiple target evaluation indicators to obtain the preset total score; and determining the software efficiency score based on the total R&D indicator score, the actual total score, and the preset total score.

[0159] Specifically, the software efficiency score satisfies the following formula sixteen: Formula Sixteen.

[0160] Where S represents the software efficiency score. Used to represent the preset indicator scoring threshold corresponding to the target evaluation indicator. The value of m represents the total actual score, and m represents the number of target evaluation indicators whose R&D indicator scores are not zero. Used to represent the preset total score.

[0161] For example, target software A and target software B are software of the same type, and the total actual score for software of this type is 100 points. All R&D indicator scores for target software A are not 0, and the sum of all R&D indicator scores for target software A is 95. Therefore, the software efficiency score for target software A is 95 points. The R&D indicator score corresponding to the target coverage of target software B is 0, and the sum of all R&D indicator scores for target software B is 90. Therefore, the software efficiency score for target software B is 94.73.

[0162] Since some software may have certain indicators without assessment data during a certain assessment period, directly deducting points from the total score for these indicators would make the evaluation results inaccurate. Therefore, calculating the software efficiency score using a conversion method can make the calculated software efficiency score more accurate.

[0163] S1034. Based on multiple software efficiency scores and target R&D coefficients, determine the R&D evaluation results.

[0164] The R&D evaluation results include: the target R&D team score.

[0165] As one possible approach, the average software efficiency score is determined based on multiple software efficiency scores; the target R&D team score is then determined based on the average software efficiency score and the target R&D coefficient.

[0166] Among them, the mean software efficiency score is the ratio between the sum of the software efficiency scores and the number of target software, the sum of the software efficiency scores is the sum of multiple software efficiency scores, and the target R&D team score is the product of the mean software efficiency score and the target R&D coefficient.

[0167] For example, the target R&D team score satisfies Formula Seventeen.

[0168] Formula 17.

[0169] As shown in S1031-S1034, by determining the software development coefficient for each target software, the level of business risk for each target software can be determined. Correcting the initial indicator scores using the software development coefficient allows the scores corresponding to the target evaluation indicators to better align with the actual situation of the target software, thereby enabling a more accurate assessment of the work efficiency of the target R&D team.

[0170] As another possible implementation, S103 can be implemented as follows: S1041. After determining the R&D indicator score corresponding to each target evaluation indicator in S1032, for each target R&D indicator whose R&D indicator score is not 0, obtain the number of first software and the number of second software.

[0171] Here, "first software" refers to the target software corresponding to the first target R&D indicator, and "first target R&D indicator" refers to the target R&D indicator whose R&D indicator score is not 0. The total number of software is either the number of first software or the number of second software, where the number of second software is the number of target software that has been launched.

[0172] As one possible implementation, the number of the second software can be determined as follows: if the number of the first software is greater than or equal to the number of the second software, then the number of the first software is taken as the total number of software; if the number of the first software is less than the number of the second software, then the number of the second software is taken as the total number of software.

[0173] S1042. For each primary target R&D indicator, based on the number of primary software and the total number of software, determine the team scoring coefficient to determine the team scoring coefficient for each primary target R&D indicator.

[0174] As one possible implementation, the team scoring coefficient satisfies Formula 18.

[0175] Formula 18.

[0176] Where K2 represents the team rating coefficient, N2 represents the number of first software programs, and M represents the total number of software programs.

[0177] S1043. For each first target R&D indicator, the team indicator score is determined based on the team scoring coefficient, the R&D indicator score of each target software, and the number of first software, so as to determine the team indicator score corresponding to each first target R&D indicator.

[0178] As one possible implementation, the team indicator score corresponding to one of the target evaluation indicators of the target R&D team satisfies Formula Nineteen.

[0179] Formula 19.

[0180] Where T represents the team metric score, Z i This is used to represent the R&D indicator score corresponding to a target evaluation metric for the target software.

[0181] For example, target R&D team 1 is responsible for developing 5 target software programs. The number of target software programs launched in the current month is 5. The number of target software programs with a non-zero R&D indicator score corresponding to the target coverage rate in the current month is 3: Software 1, Software 2, and Software 3. For the target coverage rate, Software 1 has a R&D indicator score of 4, Software 2 has a score of 5, and Software 3 has a score of 4. Therefore, the team indicator score corresponding to the target coverage rate satisfies Formula 20.

[0182] Formula 20.

[0183] Alternatively, consider a development team (team 1) responsible for developing 5 target software programs. Three of these programs are launched that month, and the number of target software programs with a non-zero R&D metric score corresponding to the target coverage rate is 5: Software 1, Software 2, Software 3, Software 4, and Software 5. For the target coverage rate, Software 1 has a R&D metric score of 4, Software 2 has a score of 5, Software 3 has a score of 4, Software 4 has a score of 4, and Software 5 has a score of 5. Therefore, the team metric score corresponding to the target coverage rate satisfies Formula 21.

[0184] Formula 21.

[0185] Alternatively, consider a development team (Team 1) responsible for developing 5 target software programs. Three of these programs are launched that month, and the number of target software programs with a non-zero R&D metric score corresponding to the target coverage rate is also three: Software 1, Software 2, and Software 3. For the target coverage rate, Software 1 has a R&D metric score of 4, Software 2 has a score of 5, and Software 3 has a score of 4. Therefore, the team metric score corresponding to the target coverage rate satisfies Formula 22.

[0186] Formula 22.

[0187] S1044. Determine the target R&D team score based on the team indicator score and target R&D coefficient corresponding to each first target R&D indicator.

[0188] As one possible approach, the average team indicator score is determined based on the team indicator score corresponding to each first target R&D indicator; the target R&D team score is then determined based on the average team indicator score and the target R&D coefficient.

[0189] Among them, the average team indicator score is the ratio between the team indicator score and the number of first target R&D indicators, the team indicator score is the sum of the team indicator scores corresponding to all first target R&D indicators, and the target R&D team score is the product of the average team indicator score and the target R&D coefficient.

[0190] As another possible implementation, multiple initial R&D team scores are determined based on the team indicator scores and target R&D coefficients corresponding to each first target R&D indicator; the target R&D team score is then determined based on these multiple initial R&D team scores. Finally, the R&D evaluation result is determined based on these multiple initial indicator scores and the target R&D coefficients. The target R&D team score is the ratio between the initial R&D team score and the number of initial R&D team scores, and the initial R&D team score is the sum of multiple initial R&D team scores.

[0191] Because the number of target software programs managed by each target R&D team varies significantly, not all target software can obtain all target evaluation metrics. In other words, for some target software, the initial score for any one of the multiple target evaluation metrics is 0, meaning the R&D metric score for that target software is 0. Therefore, by averaging the team metric scores, the calculated scores for the target R&D teams can be made more accurate, thus leading to a more accurate R&D evaluation result.

[0192] As can be seen from S101-S103, the solution provided in this application embodiment, in response to the evaluation instruction, obtains the target unit test coverage, unit test case validity, and technical debt repair rate of the target code corresponding to the target software, and determines the corresponding initial indicator score. This allows for the determination of the coverage of unit test cases to the target code, the validity of unit test cases, and the problem repair status of the target code. Consequently, it accurately determines the completion status of the target R&D team for R&D tasks such as deleting, modifying, and / or adding target functions to the target software, thereby determining the work efficiency of the target R&D team. Simultaneously, by obtaining the target R&D coefficient, the complexity of the R&D tasks can be considered, thus enabling accurate determination of the R&D evaluation results and achieving an accurate assessment of the R&D work efficiency of the target R&D team.

[0193] In some embodiments, after executing S101-S103, the R&D evaluation results of each target R&D team are monotonically sorted to obtain a first ranking report, so as to display the team ranking of each target R&D team to the user.

[0194] The monotonic ranking of the R&D evaluation results of each target R&D team can be performed in either ascending order or descending order. This application does not impose any restrictions on this.

[0195] The first ranking report includes: the ranking number of the target R&D team, the team name of the target R&D team, and the R&D evaluation results of the target R&D team.

[0196] After obtaining the team rankings, quality improvement suggestions can be made to the target R&D teams with low rankings.

[0197] In some embodiments, after performing S101-S103, the evaluation level and team type of the target R&D team are determined based on the R&D evaluation results of the target R&D team.

[0198] For example, assuming a team score out of 100: If the team score is 95 or higher, the target R&D team is rated as excellent and classified as a quality-efficiency team. If the team score is 80 or higher but less than 95, the target R&D team is rated as good and classified as a R&D quality-oriented team. If the team score is 60 or higher but less than 80, the target R&D team is rated as medium-risk and classified as a quality-medium-risk team. If the team score is less than 60, the target R&D team is rated as high-risk and classified as a quality-high-risk team.

[0199] In some embodiments, after executing S1033, the software efficiency scores of each target software in the multiple software efficiency scores are monotonically sorted to obtain a first ranking report, so as to display the software ranking of each target software in each target R&D team to the user.

[0200] The software efficiency score of each target software can be monotonically sorted in either ascending or descending order. This application does not impose any restrictions on this method.

[0201] The second ranking report includes: the target software's ranking number, the target software's name, and the target software's software efficiency score.

[0202] In some embodiments, after executing S102, the initial indicator score corresponding to each target evaluation indicator is compared with a preset initial score threshold to determine the indicator risk result.

[0203] Among them, the indicator risk results are used to reflect the efficiency of the R&D sub-tasks corresponding to the target evaluation indicators. The R&D tasks include multiple R&D sub-tasks.

[0204] As one possible approach, if the initial indicator score is less than a preset initial score threshold, the indicator risk result is determined to be low work efficiency of the R&D sub-task corresponding to the target evaluation indicator; if the initial indicator score is greater than or equal to the preset initial score threshold, the indicator risk result is determined to be normal work efficiency of the R&D sub-task corresponding to the target evaluation indicator.

[0205] If the indicator risk result shows low work efficiency in the R&D sub-tasks corresponding to the target evaluation indicator, an early warning message can be issued, and a risk analysis can be conducted on the quality issues in the R&D process related to the target evaluation indicator. Furthermore, for the different initial indicator scores of each target R&D team, example radar charts of indicator strength and weakness can be generated for analysis, providing improvement references for each target R&D team.

[0206] The foregoing primarily describes the solutions provided in the embodiments of this application from a methodological perspective. It is understood that the R&D efficiency evaluation device, in order to achieve the aforementioned functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the R&D efficiency evaluation method steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0207] This application embodiment can divide the R&D work efficiency evaluation device into functional modules or functional units according to the above method example. For example, each function can be divided into a separate functional module or functional unit, or two or more functions can be integrated into one module. The integrated module can be implemented in hardware or in software functional modules or functional units. The module or unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0208] This application provides an apparatus for evaluating the efficiency of research and development work. For example... Figure 2 As shown, the device for evaluating the efficiency of R&D work may include: a response module 201, an acquisition module 202, and a determination module 203.

[0209] Response module 201 is used to respond to evaluation instructions.

[0210] Module 202 is used to acquire multiple target evaluation metrics and target R&D coefficients. The target evaluation metrics assess the efficiency of the target R&D team in completing R&D tasks, which include deleting, modifying, and / or adding target functions to the target software. The multiple target evaluation metrics include: target code coverage, unit test case effectiveness, and technical debt repair rate. The target code is used to implement the target functions. Target coverage indicates the extent to which unit test cases cover the target code. The technical debt repair rate is the ratio between the number of fixed issues and the previous technical debt. The number of fixed issues is the difference between the previous technical debt and the current technical debt. The previous technical debt refers to issues detected by static code testing of historical code, while the current technical debt refers to issues detected by static code testing of the target code. The target code is the updated historical code. The target R&D coefficient reflects the complexity of the R&D tasks.

[0211] The determination module is used to determine the initial indicator score for each of the multiple target evaluation indicators. This module also determines the R&D evaluation result based on the multiple initial indicator scores and the target R&D coefficient. The R&D evaluation result is used to indicate the efficiency of the target R&D team in completing the R&D tasks.

[0212] In some embodiments, the device for evaluating R&D efficiency further includes a sorting module and a grouping module. The number of target R&D teams is multiple.

[0213] The determination module is used to determine the initial R&D coefficient corresponding to the target R&D team. The initial R&D coefficient is related to the amount of code of the target software provided by the target R&D team and the number of target functions corresponding to the completion of the R&D task.

[0214] The sorting module is used to monotonically sort the initial R&D coefficients corresponding to multiple target R&D teams.

[0215] The grouping module is used to group multiple initial R&D coefficients based on the sorted initial R&D coefficients, resulting in multiple monotonically sorted sets of initial R&D coefficients.

[0216] The determination module is also used to determine the target R&D coefficients corresponding to each initial R&D coefficient set based on the sequence number of each initial R&D coefficient set.

[0217] In some embodiments, the acquisition module is used to acquire the number of completed target functions, a first average code volume, a second average code volume, the number of adjustment functions, and the predicted number of completed functions. The first average code volume is the average code volume of the target R&D team, the second average code volume is the average code volume of the target R&D department to which the target R&D team belongs, the number of adjustment functions is the number of target functions that need to have their code added and / or modified, and the predicted number of functions is the number of target functions expected to be completed.

[0218] The determination module is used to determine the initial R&D coefficients based on the number of completed target functions, the first average code volume, the second average code volume, the number of adjusted functions, and the predicted number of completed functions.

[0219] In some embodiments, the initial R&D coefficients satisfy the following formula: ; in, The initial R&D coefficient is represented by N1, the number of target functions completed is represented by Lavg1, the first average code size is represented by Lavg2, the second average code size is represented by n1, the number of adjustment functions is represented by n2, and the predicted number of functions to be completed is represented by n2.

[0220] In some embodiments, there are multiple target software programs. The acquisition module is used to acquire the software development coefficient of each target software program, whereby the software development coefficient reflects the degree of business risk of the target software.

[0221] The determination module is used to determine the R&D indicator score corresponding to each target evaluation indicator for each target software based on multiple initial indicator scores and software R&D coefficients, thereby determining the multiple R&D indicator scores corresponding to each target software.

[0222] The determination module is also used to determine the software efficiency score of each target software based on multiple R&D indicator scores, thereby determining multiple software efficiency scores.

[0223] The determination module is also used to determine the R&D evaluation results based on multiple software efficiency scores and target R&D coefficients.

[0224] In some embodiments, the device for evaluating R&D efficiency further includes an addition calculation module. The determination module is also used to determine whether all scores for multiple R&D indicators are not zero.

[0225] The addition calculation module is used to add up the scores of multiple R&D indicators if none of them are 0, and obtain the software efficiency score of the target software.

[0226] The determination module is used to determine the software efficiency score of the target software based on the scores of multiple R&D indicators and the preset indicator score threshold corresponding to each target evaluation indicator if any R&D indicator score among multiple R&D indicator scores is not 0.

[0227] In some embodiments, the determining module is used to determine the total score of R&D indicators based on multiple R&D indicator scores.

[0228] The determination module is used to determine the actual total score based on the preset score threshold corresponding to the target evaluation indicator whose R&D indicator score is not 0.

[0229] The addition calculation module is used to add up the preset indicator scoring thresholds corresponding to multiple target evaluation indicators to obtain the preset total score.

[0230] The determination module is also used to determine the software efficiency score based on the total score of R&D indicators, the actual total score, and the preset total score.

[0231] In some embodiments, the software efficiency score satisfies the following formula: .

[0232] in, Used to represent R&D indicator scores n is used to represent the total score of R&D indicators, and n3 is used to represent the number of target evaluation indicators. Used to represent the preset indicator scoring threshold corresponding to the target evaluation indicator. The value of m represents the total actual score, and m represents the number of target evaluation indicators whose R&D indicator scores are not zero. Used to represent the preset total score.

[0233] Figure 3 This is a schematic diagram illustrating the structure of another R&D efficiency evaluation apparatus according to an exemplary embodiment. The R&D efficiency evaluation apparatus may include a processor 302, which executes application code to implement the R&D efficiency evaluation method of this application.

[0234] Processor 302 may be a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0235] like Figure 3 As shown, the device for evaluating R&D work efficiency may further include a memory 303. The memory 303 stores the application code that executes the scheme of this application, and its execution is controlled by the processor 302.

[0236] Memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 303 may exist independently and be connected to processor 302 via bus 304. Memory 303 may also be integrated with processor 302.

[0237] like Figure 3 As shown, the R&D efficiency evaluation device may further include a communication interface 301, wherein the communication interface 301, processor 302, and memory 303 may be coupled to each other, for example, through a bus 304. The communication interface 301 is used for information exchange with other devices, for example, supporting information exchange between the R&D efficiency evaluation device and other devices.

[0238] It should be pointed out that, Figure 3 The equipment structure shown does not constitute a limitation on the evaluation device for the efficiency of this research and development work, except... Figure 3 In addition to the components shown, the evaluation device for R&D efficiency may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0239] In actual implementation, the functions implemented by the response module 201, the acquisition module 202, and the determination module 203 can be derived by... Figure 3 The processor 302 shown calls the program code in memory 303 to implement this.

[0240] This application also provides a computer-readable storage medium storing instructions that, when executed by a processor of a computer device, enable the computer to perform the research and development efficiency evaluation method provided in the embodiments described above. For example, the computer-readable storage medium may be a memory 303 including instructions, which may be executed by a processor 302 of a computer device to complete the method. Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0241] Figure 4 A conceptual partial view of a computer program product provided in an embodiment of this application is shown schematically. The computer program product includes a computer program for executing computer processes on a computing device.

[0242] In one embodiment, the computer program product is provided using a signal bearer medium 400. The signal bearer medium 400 may include one or more program instructions that, when executed by one or more processors, can provide the above-mentioned... Figure 1 The described function or part of the function. Therefore, for example, refer to... Figure 1 In the embodiment shown, one or more features of S101-S103 can be provided by one or more instructions associated with the signal carrying medium 400. Furthermore, Figure 4 The program instructions in the document also describe example instructions.

[0243] In some examples, the signal carrying medium 400 may include a computer-readable medium 401, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video optical disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), and so on.

[0244] In some implementations, the signal carrying medium 400 may include a computer recordable medium 402, such as, but not limited to, a memory, a read / write (R / W) CD, an R / W DVD, and so on.

[0245] In some implementations, the signal carrying medium 400 may include a communication medium 403, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0246] The signal-bearing medium 400 can be transmitted by a wireless communication medium 403. One or more program instructions may be, for example, computer-executable instructions or logical implementation instructions.

[0247] In some examples, the device for evaluating the efficiency of research and development work can be configured to provide various operations, functions, or actions in response to one or more program instructions via a computer-readable medium 401, a computer-recordable medium 402, and / or a communication medium 403.

[0248] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0249] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0250] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the constituent units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0251] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0252] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, essentially, or the part that contributes to the prior art, or a complete or partial classification of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0253] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for evaluating the efficiency of research and development work, characterized in that, The method includes: In response to the evaluation instruction, multiple target evaluation indicators and target R&D coefficients are obtained. The multiple target evaluation indicators are used to evaluate the efficiency of the target R&D team in completing R&D tasks, which include: deleting, modifying and / or adding target functions of the target software; the multiple target evaluation indicators include: target code coverage, unit test case effectiveness and technical debt repair rate. Wherein, the target code is used to implement the target function; the target coverage rate is used to indicate the degree to which the unit test cases cover the target code; the technical debt repair rate is the ratio between the number of repaired technical debts and the number of previous technical debts, the number of repaired technical debts is the difference between the number of previous technical debts and the number of current technical debts, the previous technical debts are problems detected by static code testing of historical code, the current technical debts are problems detected by static code testing of the target code, and the target code is the updated historical code; the target development coefficient is used to reflect the complexity of the development task; Based on the multiple target evaluation indicators, determine the initial indicator score corresponding to each of the multiple target evaluation indicators; Based on the initial indicator scores and the target R&D coefficient, an R&D evaluation result is determined, which is used to indicate the efficiency of the target R&D team in completing R&D tasks.

2. The method according to claim 1, characterized in that, The number of target R&D teams is multiple, and the target R&D coefficient is obtained through the following methods: Determine the initial R&D coefficient corresponding to the target R&D team. The initial R&D coefficient is related to the amount of code of the target software provided by the target R&D team and the number of target functions corresponding to the completion of the R&D task. The initial R&D coefficients corresponding to the multiple target R&D teams are monotonically sorted. Based on the sorted initial R&D coefficients, multiple monotonically sorted initial R&D coefficient sets are obtained by grouping them. Based on the index of each initial R&D coefficient set, the target R&D coefficient corresponding to the initial R&D coefficient set is determined.

3. The method according to claim 2, characterized in that, The initial R&D coefficient was obtained in the following way: The number of completed target functions, the first average code volume, the second average code volume, the number of adjusted functions, and the predicted number of completed functions are obtained. The first average code volume is the average code volume of the target R&D team, the second average code volume is the average code volume of the target R&D department to which the target R&D team belongs, the number of adjusted functions is the number of target functions that need to have their code added and / or modified, and the predicted number of functions is the number of target functions expected to be completed. The initial R&D coefficient is determined based on the number of completed target functions, the first average code volume, the second average code volume, the number of adjusted functions, and the predicted number of completed functions.

4. The method according to claim 3, characterized in that, The initial R&D coefficients satisfy the following formula: ; in, The initial R&D coefficient is used to represent the initial R&D coefficient, N1 is used to represent the number of completed target functions, Lavg1 is used to represent the first average code size, Lavg2 is used to represent the second average code size, n1 is used to represent the number of adjustment functions, and n2 is used to represent the number of predicted completed functions.

5. The method according to any one of claims 1-4, characterized in that, The number of target software programs is multiple; the determination of the R&D evaluation result based on multiple initial indicator scores and the target R&D coefficient includes: Obtain the software development coefficient for each of the multiple target software programs, whereby the software development coefficient reflects the degree of business risk of the target software. For each target software, based on the multiple initial indicator scores and the software development coefficient, the development indicator score corresponding to each target evaluation indicator is determined, so as to determine the multiple development indicator scores corresponding to each target software; For each target software, a software efficiency score is determined based on the multiple R&D indicator scores, thereby determining multiple software efficiency scores; The R&D evaluation result is determined based on multiple software efficiency scores and the target R&D coefficient.

6. The method according to claim 5, characterized in that, The process of determining the software efficiency score of the target software based on the scores of the multiple R&D indicators includes: Determine whether all scores of the aforementioned R&D indicators are not 0; If all of the R&D indicator scores are not 0, then the R&D indicator scores are added together to obtain the software efficiency score of the target software. If any of the multiple R&D indicator scores is not 0, then the software efficiency score of the target software is determined based on the multiple R&D indicator scores and the preset indicator score threshold corresponding to each target evaluation indicator.

7. The method according to claim 6, characterized in that, The process of determining the software efficiency score of the target software based on the scores of the multiple R&D indicators and the preset indicator score threshold corresponding to each target evaluation indicator includes: The total score for the R&D indicators is determined based on the scores of the aforementioned multiple R&D indicators. The actual total score is determined based on the preset score threshold corresponding to the target evaluation indicator where at least one R&D indicator score is not 0. The preset score thresholds corresponding to multiple target evaluation indicators are added together to obtain the preset total score. The software efficiency score is determined based on the total score of the R&D indicators, the total actual score, and the total preset score.

8. The method according to claim 7, characterized in that, The software efficiency score satisfies the following formula: ； in, Used to represent R&D indicator scores n3 represents the total score of the R&D indicators, and n3 represents the number of the target evaluation indicators. This is used to represent the preset indicator scoring threshold corresponding to the target evaluation indicator. The value of m represents the total actual score, and m represents the number of target evaluation indicators whose R&D indicator scores are not zero. Used to represent the preset total score.

9. A computer-readable storage medium storing instructions, characterized in that, When the computer executes the instruction, the computer performs the method as described in any one of claims 1-8.

10. A computer program product containing instructions, characterized in that, When the instructions are executed by the computing device, the computing device performs the method as described in any one of claims 1-8.

Images