A program evaluation method, system, storage medium and terminal device

Abstract

Embodiments of the present application disclose a program evaluation method and system, a storage medium and a terminal device, and apply to the technical field of information processing. The program evaluation system can develop information of requirements, test information and information of major defects of a target application program according to the requirements of the target application program, determine the incompleteness of the requirements, the remaining incompleteness and the incompleteness of the major defects of the function of the target application program, and further determine the overall incompleteness of the target application program. In this way, the overall incompleteness of the application program can well represent the progress and maturity of the application program, and the evaluation of the progress of the target application program is unified to the determination of the incompleteness of each dimension, so that the complexity of the evaluation of the application program is greatly reduced, and the evaluation of the application program is quickly realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information processing, in particular to a program evaluation method, system, storage medium and terminal device. BACKGROUND

[0002] The existing software maturity evaluation technology mainly includes evaluation models such as software capability maturity model (CMM), organizational project management maturity model (OPM3) and Kerzner project management maturity model (K-PMMM), which can effectively guide the work of software in the development process by evaluating various aspects of the entire project.

[0003] The existing software maturity evaluation method uses various maturity models that consider a large number of factors, and although the evaluation results are relatively complete and accurate, the resource investment is also relatively large, which is not suitable for scenarios that require rapid evaluation of project progress. SUMMARY

[0004] The embodiments of the present application provide a program evaluation method, system, storage medium and terminal device, which can quickly evaluate an application program.

[0005] In one aspect, the embodiments of the present application provide a program evaluation method, which includes:

[0006] Obtaining requirement development information of requirements involved in a target application program, and determining a requirement incompleteness degree of the target application program according to the requirement development information;

[0007] Obtaining test information of the target application program when testing, and determining a remaining incompleteness degree of the target application program according to the test information;

[0008] Obtaining information of major defects of the target application program, and determining a functional major defect incompleteness degree of the target application program according to the information of major defects; the major defects are defects with a preset defect level;

[0009] According to the requirement incompleteness degree, the remaining incompleteness degree and the functional major defect incompleteness degree, determining an overall incompleteness degree of the target application program, and performing progress analysis on the target application program.

[0010] In another aspect, the embodiments of the present application provide a program evaluation system, which includes:

[0011] a requirement unit configured to acquire requirement development information of requirements involved in a target application program, and determine a requirement incompleteness degree of the target application program according to the requirement development information;

[0012] a legacy unit configured to acquire test information of the target application program when tested, and determine a legacy incompleteness degree of the target application program according to the test information;

[0013] a critical defect unit configured to acquire information of critical defects of the target application program, and determine a functional critical defect incompleteness degree of the target application program according to the information of the critical defects; the critical defects are defects with a preset defect level;

[0014] a whole evaluation unit configured to determine a whole incompleteness degree of the target application program according to the requirement incompleteness degree, the legacy incompleteness degree and the functional critical defect incompleteness degree, and perform progress analysis on the target application program.

[0015] In another aspect, the embodiment of the present application further provides a computer readable storage medium storing a plurality of computer programs, the computer programs being adapted to be loaded and executed by a processor to implement the program evaluation method according to the first aspect of the embodiment of the present application.

[0016] In another aspect, the embodiment of the present application further provides a terminal device including a processor and a memory.

[0017] The memory is configured to store a plurality of computer programs, the computer programs being used to be loaded and executed by the processor to implement the program evaluation method according to the first aspect of the embodiment of the present application; and the processor is configured to implement each of the plurality of computer programs.

[0018] It can be seen that, in the method of the embodiment, the program evaluation system can determine the requirement incompleteness degree, the legacy incompleteness degree and the functional critical defect incompleteness degree of the target application program according to the requirement development information, the test information and the information of the critical defects of the target application program, and then determine the whole incompleteness degree of the target application program. In this way, the progress and maturity of the application program can be well represented by the whole incompleteness degree of the application program, and the evaluation of the progress of the target application program is unified to the determination of the incompleteness degrees of each dimension, so that the complexity of the evaluation of the application program is greatly reduced, and the evaluation of the application program is quickly realized. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to make the technical solutions of the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only aim to some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative effort based on these drawings.

[0020] Figure 1 is a schematic diagram of a program evaluation method provided by an embodiment of the present application;

[0021] Figure 2 is a flowchart of a program evaluation method provided by an embodiment of the present application;

[0022] Figure 3 is a flowchart of a program evaluation method provided by an application embodiment of the present application;

[0023] Figure 4 is a flowchart of determining the degree of uncompleted requirements in an application embodiment of the present application;

[0024] Figure 5 is a flowchart of determining the degree of uncompleted legacy in an application embodiment of the present application;

[0025] Figure 6 is a flowchart of determining the degree of uncompleted major functional defects in an application embodiment of the present application;

[0026] Figure 7 is a flowchart of determining the defect level in an application embodiment of the present application;

[0027] Figure 8 is a schematic diagram of a distributed system to which the program evaluation method in another application embodiment of the present application is applied;

[0028] Figure 9 is a schematic diagram of a block structure in another application embodiment of the present application;

[0029] Figure 10 is a logical structure schematic diagram of a program evaluation system provided by an embodiment of the present application;

[0030] Figure 11 is a logical structure schematic diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0032] The terms "first", "second", "third", "fourth" and the like in the description and claims of the present application, and above drawings (if any) are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in other than the order illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a list of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0033] The embodiments of the present application provide a program evaluation method, which mainly evaluates the unfinished degree of functions of any application program (i.e. target application program) in the development process, and then analyzes the progress of the target application program. Specifically, as shown in Figure 1 The program evaluation system can realize the evaluation of the target application program by the following steps:

[0034] Obtaining the requirement development information of the requirements involved in the target application program, and determining the unfinished degree of requirements of the target application program according to the requirement development information;

[0035] Obtaining the test information of the target application program when testing, and determining the remaining unfinished degree of the target application program according to the test information;

[0036] Obtaining the information of the major defects of the target application program, and determining the unfinished degree of the major functional defects of the target application program according to the information of the major defects; the major defects are defects with preset defect levels;

[0037] According to the unfinished degree of requirements, the remaining unfinished degree and the unfinished degree of major functional defects, the overall unfinished degree of the target application program is determined to analyze the progress of the target application program.

[0038] In this way, the progress and maturity of the application program can be well represented by the overall unfinished degree of the application program, and the evaluation of the progress of the target application program is unified to the determination of the unfinished degree of each dimension, so that the complexity of the application program evaluation is greatly reduced, and the evaluation of the application program is realized quickly.

[0039] The embodiment of the application provides a program evaluation method, mainly a method executed by a program evaluation system, a flowchart is as shown in Figure 2 The embodiment of the application provides a program evaluation method, mainly a method executed by a program evaluation system, a flowchart is as shown in

[0040] In step 101, requirement development information of requirements involved in the target application program is acquired, and the unfinished degree of the requirements of the target application program is determined according to the requirement development information.

[0041] It can be understood that any application program (i.e., the target application program) needs to be developed before being online, specifically, mainly creating, function development, testing and updating and the like of the target application program. In this process, the user (mainly the development user) can initiate the flow of the embodiment through the program evaluation system to evaluate the overall unfinished degree (project issue value, PIV) of the target application program, so as to analyze the development progress of the target application program and guide the development process of the target application program.

[0042] When the flow of the embodiment is initiated, information related to the development of the target application program needs to be acquired first, and the unfinished degrees of multiple dimensions are determined based on the information. In the embodiment, the unfinished degrees of the following several dimensions are mainly involved: requirements, defects and test cases, wherein: the requirements refer to a set of functions that the target application program needs to implement, the defects refer to various problems that cannot meet the normal operation of the target application program, and the test cases contain test criteria for testing whether various function scenarios in the target application program meet the requirements.

[0043] Correspondingly, the unfinished degree determined for these dimensions mainly includes: functional requirement issue value (FIV), remaining issue value (RIV) and function loss issue value (LIV), wherein the functional requirement issue value is used to indicate the degree of unfinished development of all functions involved in the target application program; the remaining issue value is used to indicate the degree of unfinished processing of discovered defects after the development of the functions of the target application program and the testing of the target application program based on the test results; and the function loss issue value is used to indicate the degree of unfinished processing of defects with preset levels based on the test results after the testing of the target application program.

[0044] In this way, the progress analysis of the target application program is unified to the overall unfinished degree of the target application program, which can reduce the complexity of the progress analysis of the application program.

[0045] Specifically, the requirement development information of the target application program involved in the determination of the functional requirement issue value in this step 101 mainly refers to the information generated in the development process of each requirement (i.e., function) of the target application program, and can specifically include but is not limited to any of the following information: development process state of the requirement, requirement priority and development workload, etc. The development process state can include but is not limited to the following states: newly created, to be developed, to be tested, tested, closed, changed and deleted, etc.

[0046] In this way, when determining the overall functional requirement issue value of the target application program, the first requirement weight value and the second requirement weight value can be set for the development process state and the requirement priority of each requirement included in the target application program, and the third requirement weight value can be set for the development workload; the sub-unfinished degree of each requirement is determined according to the first requirement weight value, the second requirement weight value and the third requirement weight value corresponding to each requirement, such as the sub-unfinished degree of any requirement being the product of the first requirement weight value, the second requirement weight value and the third requirement weight value of the requirement; and the overall functional requirement issue value of the target application program is determined according to the sub-unfinished degree of each requirement, such as the overall functional requirement issue value being the sum of the sub-unfinished degrees of each requirement.

[0047] In step 102, test information of the target application program during testing is obtained, and the remaining issue value of the target application program is determined according to the test information.

[0048] Specifically, the test information of the target application used in determining the remaining unfinished degree in the present step 102 mainly refers to information generated in the process of testing and repairing each requirement (i.e., function) after the requirement is developed in the target application, and can specifically include but is not limited to any of the following information: information of test cases involved in testing the target application (such as classification, total number, etc.), information of defects found in the testing process, and information of defects handled, etc. The handled defects refer to defects that have been repaired, including defects in the closed, suspended, and verified states.

[0049] The program evaluation system can first divide all test cases involved in the target application into multiple groups, such as dividing the test cases according to the modules included in the target application, or dividing the test cases according to the attributes of each test case, etc.

[0050] In this way, in determining the remaining unfinished degree, the overall test completion rate of the target application can be first determined according to the completion rate of the test cases of each group and the test case proportion; the remaining defect number of the target application can be determined according to the overall test completion rate, the number of defects found, and the number of defects handled, such as the value obtained by subtracting the number of defects handled from the product of the number of defects found and the overall test completion rate; the first remaining weight value based on the handling state and the second remaining weight value based on the defect level are set for the remaining defects corresponding to the test cases of each group; and the remaining unfinished degree of the target application is determined according to the number of remaining defects of the test cases, the first remaining weight value, and the second remaining weight value, such as the product of the number of remaining defects, the first remaining weight value, and the second remaining weight value.

[0051] Further, in determining the overall test completion rate, the classification test completion rate of each group can be determined according to the completion rate of the test cases of each group and the test case proportion, such as the classification test completion rate of any group being the product of the completion rate of the test cases of the group and the test case proportion, and the test case proportion being the ratio of the number of test cases of the group to the total number of test cases; and the sum of the classification test completion rates of each group is taken as the overall test completion rate of the target application. The completion rate of the test cases of any group refers to the ratio of the test cases that have completed the test among all the test cases of the corresponding group.

[0052] Further, when setting the second legacy weight value based on the defect level for the legacy defect corresponding to each of the test cases of the above-mentioned groups, the defect level of the legacy defect needs to be determined first, specifically, the defect level of the legacy defect needs to be determined according to the product of the weight values corresponding to the reproduction probability, the user discovery probability and the impact degree of the legacy defect, for example, the defect level is determined according to the product of the weight values corresponding to the reproduction probability, the user discovery probability and the impact degree; and then the second legacy weight value is set according to the defect level. The reproduction probability is used to represent the probability of repeated occurrence of the legacy defect; the user discovery probability is used to represent the probability of being discovered when the legacy defect is not processed and left to the user usage scenario; and the impact degree is used to represent the degree of adverse impact caused by the legacy defect.

[0053] It should be noted that when determining the defect level of a defect, in addition to the above-mentioned reproduction probability, user discovery probability and impact degree and the like, other parameters such as time priority state and the like can also be combined, wherein if a certain application program needs to be shipped urgently, then the time priority state is urgent (Urgent), and if the application program does not need to be shipped urgently, then the time priority state is normal (Normal).

[0054] Step 103, obtaining information of critical defects of the target application program, and determining the function critical defect incompleteness of the target application program according to the information of the critical defects, wherein the critical defects are defects with a preset level.

[0055] Specifically, the information of the critical defects used when determining the function critical defect incompleteness in this step 103 mainly refers to the information of the preset level critical defects found after testing the functions developed for the target application program, and in the embodiment of the present application, the preset level mainly refers to the urgency or emphasis degree that needs to be solved, and can include at least one of the following information: the number of critical defects and the processing state of each critical defect and the like. The processing state can include: new, to be repaired, to be released, to be verified or suspended and the like.

[0056] In this way, when determining the function critical defect incompleteness, a corresponding processing weight value can be set for each critical defect according to the processing state of each critical defect, and then the function critical defect incompleteness can be determined according to the processing weight value and the number of critical defects, for example, the product of each critical defect processing weight value and the number of critical defects is added to obtain an added value as the function critical defect incompleteness.

[0057] It should be noted that there is no absolute sequence relationship between the above-mentioned steps 101 to 103, which can be executed simultaneously or sequentially. Figure 2 The above-mentioned only shows one specific implementation manner.

[0058] Step 104, determining the overall unfinished degree of the target application according to the requirement unfinished degree, the remaining unfinished degree and the function major defect unfinished degree, to analyze the progress of the target application.

[0059] Specifically, the weighted sum of the requirement unfinished degree, the remaining unfinished degree and the function defect unfinished degree can be taken as the overall unfinished degree.

[0060] Further, the program evaluation system can store the information of the requirements and defects involved in the target application respectively during the execution of the above steps 101 to 104, and then establish the corresponding relationship between the requirements and the defects through a relationship matrix. The relationship matrix can include multiple elements, each element corresponding to the relationship between a requirement and a defect, i.e. the relationship of whether associated or not, and the corresponding requirement and defect can be located through the position of each element in the relationship matrix.

[0061] In this way, the corresponding relationship between the requirements and the defects is set, and the defect level of each defect can be determined based on the corresponding relationship. If the element at a certain position in the relationship matrix indicates that a requirement and a defect are associated, a higher defect level can be set for the defect. If the element at a certain position in the relationship matrix indicates that a requirement and a defect are not associated, a lower defect level can be set for the defect.

[0062] As can be seen, in the method of the embodiment, the program evaluation system can determine the requirement unfinished degree, the remaining unfinished degree and the function major defect unfinished degree of the target application according to the requirement development information, the test information and the major defect information of the target application, and then determine the overall unfinished degree of the target application. In this way, the overall unfinished degree of the application program can well represent the progress and maturity of the application program, and the evaluation of the progress of the target application is unified to the determination of the unfinished degree of each dimension, so that the complexity of the evaluation of the application program is greatly reduced, and the evaluation of the application program is realized quickly.

[0063] The program evaluation method of the embodiment will be described below with a specific application example. As shown in FIG. 1, the method of the embodiment can include the following steps: Figure 3

[0064] Step 201, for any application program being developed (i.e. the target application), obtaining the requirement development information of the target application, which can specifically include the development process state of the requirement , the requirement priority and the development workload T, etc.

[0065] Step 202, determining the requirement unfinished degree FIV of the target application according to the requirement development information of the target application, which can reflect the development of the customer requirement. Specifically, as shown in FIG. 2, the requirement unfinished degree FIV of the target application can be determined according to the requirement development information of the target application, and the requirement unfinished degree FIV of the target application can be calculated according to the following formula:​Figure 4 In determining the degree of unfulfillment of requirements, the method can specifically include the following steps:

[0066] A1, based on the development process state of each requirement of the target application program Set the first requirement weight value of each requirement When, as shown in Table 1 below:

[0067]

[0068] Table 1

[0069] A2, based on the requirement priority of each requirement of the target application program Set the second requirement weight value of each requirement When, the second requirement weight value is mainly set according to the impact of the requirement on the customer program, and the requirement priority can be represented by the following formula 1-1: The range of the requirement priority is set to m+1:

[0070] (1-1)

[0071] A3, based on the development workload T to set the third requirement weight value When, as shown in the following formula 1-2, wherein, is the calculation weight:

[0072] (1-2)

[0073] A4, according to the first requirement weight value , the second requirement weight value and the third requirement weight value of a requirement, determine the sub-unfulfillment degree of the requirement , which can be represented by the following formula 1-3:

[0074] (1-3)

[0075] A5, according to the sub-unfulfillment degree of each requirement Determine the requirement unfulfillment degree FIV of the target application program as a whole, which can first determine the requirement unfulfillment degree of each module included in the target application program according to the following formula 1-4: Including: the sum of the sub-unfulfillment degrees of the requirements involved in each module; then according to the following formula 1-5, the sum of the requirement unfulfillment degrees of each module is taken as the requirement unfulfillment degree FIV of the target application program as a whole:

[0076] (1-4)

[0077] (1-5)

[0078] It should be noted that, since the same requirements can be involved between two modules designed by the target application program, it is more accurate to calculate the uncompleted degree of each module first, and then calculate the uncompleted degree of the target application program as a whole.

[0079] In step 203, test information of the target application program during testing is obtained, which can specifically include information of test cases involved in testing of the target application program, information of defects found during testing, and information of defects processed, etc.

[0080] In step 204, the residual uncompleted degree RIV of the target application program is determined according to the test information of the target application program. Specifically, as shown in FIG. 2, when determining the residual uncompleted degree, it can specifically include: Figure 5

[0081] B1, the test cases are divided into multiple groups, and the product of the test completion rate (TCR) of each group and the test case proportion is taken as the classification test completion rate of each group. The test case proportion is the ratio of the number of test cases involved in each group to the total number of test cases (TCA).

[0082] B2, the sum of the classification test completion rates of each group is taken as the overall test completion rate of the target application program , which can be specifically shown in the following formula 2-1. The grouping of test cases can be based on the modules (denoted as subfunc) included in the target application program, or based on the attributes (denoted as spe) of the test cases.

[0083] (2-1)

[0084] It should be noted that, since the same requirements can be involved between two modules designed by the target application program, it is more accurate to calculate the uncompleted degree of each module first, and then calculate the uncompleted degree of the target application program as a whole. The classification test completion rate of any group can be represented by the following formula 2-2, that is, the classification test completion rate of the group is the sum of the classification test completion rates of the test cases of each attribute in the group:

[0085] (2-2)

[0086] B3, according to the overall test completion rate ​The number of found issues (FIA) and processed issues (PIA) are used to determine the number of remaining issues (RIA) in the target application. Processed issues can be represented by formulas 2-3 below:

[0087] (2-3)

[0088] B4. Set the remaining defects based on processing status for the test cases of each of the above groups. First legacy weight value and based on defect level Second legacy weight value Specifically, when setting the first legacy weight value, it can be set according to Table 2 below, and the second legacy weight value can be set using Formula 2-4 below:

[0089]

[0090] Table 2

[0091] (2-4)

[0092] B5. Based on the number of remaining defects in the test cases, the first legacy weight value, and the second legacy weight value, determine the degree of incompleteness of the target application. Specifically, this can be illustrated by Formula 2-5 below:

[0093] (2-5)

[0094] Step 205: Obtain information on critical defects in the target application, which may include the number of critical defects (function loss issue amount, LIA) and the processing status of each critical defect.

[0095] Step 206: Determine the Functional Critical Flaw Incompleteness (LIV) of the target application based on the critical flaw information. Specifically, as follows... Figure 6 As shown, determining the degree of incompleteness of a major functional defect can specifically include:

[0096] C1. Assign corresponding processing weight values ​​to each critical defect based on its processing status. Since critical defects have a significant impact on the target application, the processing weight value set will be larger than the second legacy weight value set in the above steps. Specifically, it can be set as shown in Table 3 below:

[0097]

[0098] Table 3

[0099] C2, according to the processing weight value and the number of critical defects The function critical defect incompleteness LIV is determined, which can be expressed by the following formula 3:

[0100] (3)

[0101] Step 207, according to the requirement incompleteness FIV, the remaining incompleteness RIV and the function critical defect incompleteness LIV, the overall incompleteness PIV of the target application program is determined to analyze the progress of the target application program. The smaller the overall incompleteness is, the higher the maturity of the target application program is, and the better the quality is. It can be expressed by the following formula 4: and The sum of the above can be 1:

[0102] (4)

[0103] Correspondingly, the overall incompleteness of each module (denoted as subfunc) included in the target application program can be determined by the same algorithm , which is shown in the following formula 5:

[0104] (5)

[0105] It can be seen that the evaluation of the target application program in the method of the embodiment is unified to the calculation of the incompleteness of multiple dimensions of the target application program, which can quickly complete the progress evaluation of the target application program and provide effective technical support for the progress control of the target application program at any time.

[0106] It should be noted that the program evaluation system will involve determining the defect level of each defect of the target application program in the process of executing the above steps. In actual application, the defect level can be determined in the following ways, but not limited to:

[0107] (1) The time priority state of the target application program is determined first, such as the time priority state is urgent, and the corresponding defect level can be directly determined as a higher level. If the time priority state is ordinary, the defect level can be determined in other ways.

[0108] (2) The defect level of the corresponding defect is determined according to the defect probability, the user discovery probability and the influence degree of the target application program. As shown in the following formula 6, the following steps can be used to determine the defect level: Figure 7

[0109] ​​D1. Based on the probability of defect recurrence Determine the corresponding weight values .

[0110] Among them, the recurrence probability This can be expressed by the following formula 6-1, which is the ratio of the number of times the defect is reproduced, R', to the total number of test cases, R, executed to reproduce the defect:

[0111] (6-1)

[0112] In this way, the probability of recurrence will be Divide the data into k sub-intervals, and assign corresponding weight values ​​based on the recurrence probability of each sub-interval. Specifically, this can be represented by the following formula 6-2. It can be seen that the greater the recurrence probability, the greater the corresponding weight value:

[0113] (6-2)

[0114] D2. Determine the corresponding weight value based on the probability that the defect will be discovered by users. This weight value It can represent the probability of a defect being discovered when it is left in the user scenario. Specifically, it can be reflected by the type of test case and the number of test steps when the defect is discovered.

[0115] Specifically, the number of test steps (steps) of the test cases used when a defect is discovered is divided into l sub-intervals, and a corresponding weight value is set according to the number of test steps in each sub-interval. Specifically, this can be represented by the following formula 6-3. It can be seen that the larger the number of test steps, the smaller the corresponding weight value:

[0116] (6-3)

[0117] D3. Determine the corresponding weight value based on the defect impact degree (SVR). Specifically, this can be represented by the following formula 6-4, where the influence degree (SVR) can be divided into 0 levels based on practical experience. It is evident that the greater the influence degree, the smaller the corresponding weight value.

[0118] (6-4)

[0119] D4. Based on the weight values ​​corresponding to the recurrence probability, the probability of being discovered by users, and the impact, determine the defect level score for the corresponding defect. Specifically, this can be represented by the following formula 6-5:

[0120] (6-5)

[0121] Further, when the defect level score is obtained In order to facilitate processing, an indication value Pn of the defect level can be obtained according to the defect level score, which can be expressed by the following formula 6-6, and it can be seen that the higher the defect level score, the smaller the indication value Pn of the defect level, and when the indication value Pn is 0, the higher the defect level, indicating that the defect needs to be processed with a higher priority level:

[0122] (6-6)

[0123] (3) The program evaluation system can establish the correspondence between the requirements and the defects in the target application program through the relationship matrix, and determine the defect level of each defect based on the correspondence. When a requirement is associated with a defect, the defect level of the defect is higher.

[0124] Wherein, each element in the relationship matrix can be expressed by the following formula 7:

[0125] (7)

[0126] The following will illustrate the program evaluation method in the application with another specific application example. The program evaluation system in the embodiment of the application is mainly a distributed system 100, which can include a client 300 and a plurality of nodes 200 (any form of computing device in the access network, such as a server, a user terminal), and the client 300 and the nodes 200 are connected through network communication.

[0127] Taking the distributed system as a blockchain system, referring to Figure 8 is an optional structural schematic diagram of the distributed system 100 applied to the blockchain system provided by the embodiment of the application, which is formed by a plurality of nodes 200 (any form of computing device in the access network, such as a server, a user terminal) and a client 300, and the nodes form a point-to-point (P2P, Peer To Peer) network, and the P2P protocol is an application layer protocol running on the transmission control protocol (TCP, Transmission Control Protocol) protocol. In the distributed system, any machine such as a server or a terminal can join to become a node, and the node includes a hardware layer, an intermediate layer, an operating system layer and an application layer.

[0128] Referring to Figure 8 The functions of each node in the blockchain system are shown, and the functions involved include:

[0129] 1) Routing, a basic function of the node, used to support communication between nodes.

[0130] In addition to the routing function, the node can also have the following functions:

[0131] 2) Application, for deployment in the blockchain, to implement a specific business according to actual business needs, record data related to the implementation of the function to form a record data, carry a digital signature in the record data to represent the source of the task data, send the record data to other nodes in the blockchain system, and when the other nodes verify the source and integrity of the record data successfully, add the record data to the temporary block.

[0132] For example, the application implements a business including code that implements a program evaluation function, which mainly includes:

[0133] Obtain the requirement development information of the requirements involved in the target application program, and determine the requirement uncompleted degree of the target application program according to the requirement development information; obtain the test information of the target application program during testing, and determine the remaining uncompleted degree of the target application program according to the test information; obtain the information of the major defects of the target application program, and determine the functional major defect uncompleted degree of the target application program according to the information of the major defects; the major defects are defects with a preset level of defect level; and determine the overall uncompleted degree of the target application program according to the requirement uncompleted degree, the remaining uncompleted degree and the functional major defect uncompleted degree, to analyze the progress of the target application program.

[0134] 3) Blockchain, including a series of blocks (Block) connected in time sequence according to the time of generation, once a new block is added to the blockchain, it will not be removed, and the block records the record data submitted by the node in the blockchain system.

[0135] Referring to Figure 9 The block structure provided by the embodiment of the application is shown in an optional schematic diagram, each block includes the hash value of the transaction record stored in the block (the hash value of the block) and the hash value of the previous block, and the blocks are connected by the hash values to form a blockchain. In addition, the block can also include information such as the time stamp when the block is generated. The blockchain is essentially a decentralized database, which is a series of data blocks associated by using cryptographic methods, each data block contains related information, which is used to verify the validity (anti-fake) of the information and generate the next block.

[0136] The embodiment of the application also provides a program evaluation system, and a structural schematic diagram thereof is shown in Figure 10 As shown in the figure, the program evaluation system can specifically include:

[0137] The requirement unit 10 is configured to obtain requirement development information of requirements involved in a target application program, and determine a requirement uncompleted degree of the target application program according to the requirement development information.

[0138] a legacy unit 11, configured to acquire test information of the target application program at a test time, and determine a legacy incompleteness of the target application program according to the test information;

[0139] a critical defect unit 12, configured to acquire information of a critical defect of the target application program, and determine a functional critical defect incompleteness of the target application program according to the information of the critical defect; the critical defect is a defect with a preset defect level;

[0140] an overall evaluation unit 13, configured to determine an overall incompleteness of the target application program according to the requirement incompleteness determined by the requirement unit 10, the legacy incompleteness determined by the legacy unit 11, and the functional critical defect incompleteness determined by the critical defect unit 12, to perform progress analysis on the target application program.

[0141] In a specific implementation process, the requirement unit 10 is specifically configured to: if the requirement development information includes: a plurality of requirements included in the target application program, and development process states, requirement priorities, and development workloads corresponding to the plurality of requirements respectively; set first requirement weight values and second requirement weight values for the development process states and the requirement priorities of each requirement of the target application program respectively, and set a third requirement weight value for the development workload; determine a sub-incompleteness of each requirement according to the first requirement weight value, the second requirement weight value, and the third requirement weight value corresponding to the requirement; and determine the requirement incompleteness of the target application program according to the sub-incompleteness of each requirement.

[0142] The sub-incompleteness of any requirement includes a product of the first requirement weight value, the second requirement weight value, and the third requirement weight value of the corresponding requirement; and the requirement incompleteness of the target application program includes a sum of the sub-incompleteness of each requirement.

[0143] The legacy unit 11 is specifically configured to: if the test information of the target application program at the test time includes: information of a plurality of grouping test cases involved in the test of the target application program, a number of discovered defects, and a number of processed defects; determine an overall test completion rate of the target application program according to a completion rate of each grouping test case and a test case proportion; determine a number of legacy defects of the target application program according to the overall test completion rate, the number of discovered defects, and the number of processed defects; set a first legacy weight value based on a processing state and a second legacy weight value based on a defect level for a legacy defect corresponding to the test case; and determine the legacy incompleteness of the target application program according to the number of legacy defects, the first legacy weight value, and the second legacy weight value.

[0144] The legacy unit 11 is specifically configured to determine a classification test completion rate of each group according to the completion rate and test case proportion of the test case of each group, and take the sum of the classification test completion rates of the groups as the overall test completion rate when determining the overall test completion rate of the target application according to the completion rate of the test case of each group and the test case proportion. Further, the legacy unit 11 is specifically configured to determine the legacy defect number of the target application according to the overall test completion rate, the number of discovered defects and the number of processed defects, and the legacy defect number includes a value obtained by subtracting the number of processed defects from the ratio of the number of discovered defects to the overall test completion rate.

[0145] Further, the legacy unit 11 is specifically configured to determine a defect level of the legacy defect according to the recurrence probability, the probability of being discovered by a user and the impact degree of the legacy defect when setting a second legacy weight value based on the defect level for the legacy defect corresponding to the test case, and set the second legacy weight value for the legacy defect according to the defect level.

[0146] The critical defect unit 12 is specifically configured to set a corresponding processing weight value for each critical defect according to the processing state of each critical defect if the information of the critical defect includes the number of critical defects and the processing state of each critical defect, and determine the function critical defect incompleteness degree according to the processing weight value and the number of critical defects.

[0147] Further, the program evaluation system in the embodiment can further include a corresponding relationship unit 14 configured to store information of requirements and defects involved in the target application respectively, and establish a corresponding relationship between the requirements and the defects through a relationship matrix.

[0148] In the program evaluation system in the embodiment, the requirement unit 10, the legacy unit 11 and the critical defect unit 12 can respectively determine the requirement incompleteness degree, the legacy incompleteness degree and the function critical defect incompleteness degree of the target application according to the requirement development information, the test information and the information of the critical defect of the target application, and then the overall evaluation unit 13 determines the overall incompleteness degree of the target application. In this way, the progress and maturity of the application program can be well represented by the overall incompleteness degree of the application program, and the evaluation of the progress of the target application is unified to the determination of the incompleteness degree of each dimension, so that the complexity of the evaluation of the application program is greatly reduced, and the evaluation of the application program is quickly realized.

[0149] The embodiment of the application further provides a terminal device, and a structure diagram thereof is as shown in Figure 11As shown, the terminal device can have a large difference due to different configurations or performances, and can include one or more central processing units (CPUs) 20 (for example, one or more processors) and a memory 21, one or more storage media 22 (for example, one or more mass storage devices) storing one or more application programs 221 or data 222. The memory 21 and the storage medium 22 can be temporary storage or persistent storage. The program stored in the storage medium 22 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the terminal device. Further, the central processing unit 20 can be configured to communicate with the storage medium 22 and execute a series of instruction operations in the storage medium 22 on the terminal device.

[0150] Specifically, the application program 221 stored in the storage medium 22 includes a program evaluation application program, and the program can include the requirement unit 10, the legacy unit 111, the critical defect unit 12, the overall evaluation unit 13 and the corresponding relationship unit 14 in the program evaluation system described above, which will not be described here. Further, the central processing unit 20 can be configured to communicate with the storage medium 22 and execute a series of operations corresponding to the program evaluation application program stored in the storage medium 22 on the terminal device.

[0151] The terminal device can also include one or more power supplies 23, one or more wired or wireless network interfaces 24, one or more input / output interfaces 25, and / or one or more operating systems 223, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0152] The steps performed by the program evaluation system in the method embodiments described above can be based on the program evaluation system Figure 11 The structure of the terminal device as shown.

[0153] Further, another aspect of the embodiments of the present application also provides a computer readable storage medium storing a plurality of computer programs, the computer programs being adapted to be loaded and executed by a processor to perform the program evaluation method performed by the program evaluation system as described above.

[0154] Another aspect of the embodiments of the present application also provides a terminal device including a processor and a memory;

[0155] The memory is configured to store a plurality of computer programs, the computer programs being used to be loaded and executed by the processor to perform the program evaluation method performed by the program evaluation system as described above; and the processor is configured to implement each computer program in the plurality of computer programs.

[0156] In addition, according to an aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the program evaluation method provided in the various optional implementation manners.

[0157] A person of ordinary skill in the art can understand that all or part of the steps in the above-mentioned various methods of the embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer readable storage medium, which can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0158] The above provides a detailed description of the program evaluation method, system, storage medium and terminal device provided by the embodiments of the present application. The principle and implementation manner of the present application are described by applying specific examples in this paper. The above embodiment is only used to help understand the method of the present application and its core idea. Meanwhile, for a person of ordinary skill in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A program evaluation method, characterized in that, include: Obtain the requirement development information of the target application, and determine the degree of incompleteness of the requirements of the target application based on the requirement development information; Obtain the test information of the target application during testing, and determine the degree of incompleteness of the target application based on the test information; wherein, the degree of incompleteness is used to indicate the extent to which the discovered defects have not been addressed after the target application has been developed and tested, based on the test results. Obtain information on critical defects in the target application, and determine the degree of incompleteness of critical functional defects in the target application based on the information on critical defects; the critical defects are defects with a preset defect level. Based on the unfulfilled requirements, unfulfilled legacy requirements, and unfulfilled major functional defects, the overall unfulfilledness of the target application is determined in order to perform a progress analysis on the target application. The test information of the target application during testing includes: information on test cases for multiple groups involved in testing the target application, the number of defects found, and the number of defects processed; The step of determining the remaining incompleteness of the target application based on the test information specifically includes: determining the classification test completion rate of each group based on the completion rate and proportion of test cases for each group; wherein, the classification test completion rate of any group is the product of the completion rate of test cases and the proportion of test cases for that group; The sum of the classification test completion rates of each group is taken as the overall test completion rate; Based on the overall test completion rate, the number of discovered defects, and the number of processed defects, the number of remaining defects in the target application is determined; wherein, the number of remaining defects includes the value obtained by subtracting the number of processed defects from the ratio of the number of discovered defects to the overall test completion rate; Set a first legacy weight value based on the processing status and a second legacy weight value based on the defect level for the legacy defects corresponding to the test cases; The degree of incompleteness of the target application is determined based on the number of remaining defects, the first remaining weight value, and the second remaining weight value; wherein the degree of incompleteness includes the product of the number of remaining defects, the first remaining weight value, and the second remaining weight value.

2. The method as described in claim 1, characterized in that, The requirement development information includes: the development process status, requirement priority, and development workload corresponding to the multiple requirements included in the target application; The step of determining the degree of incompleteness of the requirements for the target application based on the requirements development information specifically includes: A first requirement weight value and a second requirement weight value are set for the development process status and requirement priority of each requirement of the target application, and a third requirement weight value is set for the development workload. Based on the first demand weight value, the second demand weight value, and the third demand weight value corresponding to each demand, the sub-incompleteness of each demand is determined respectively; The requirement incompleteness of the target application is determined based on the sub-incompleteness of each requirement.

3. The method as described in claim 2, characterized in that, The sub-incompleteness of any requirement includes the product of the first requirement weight value, the second requirement weight value, and the third requirement weight value of the corresponding requirement; The incompleteness of the target application's requirements includes the sum of the incompleteness of each of the individual requirements.

4. The method as described in claim 1, characterized in that, To assign a second legacy weight value based on the defect level to the legacy defects corresponding to the test cases, specifically including: The defect level of the legacy defect is determined based on its recurrence probability, user discovery probability, and impact. A second legacy weight value is set for the legacy defect based on the defect level.

5. The method according to any one of claims 1 to 3, characterized in that, The information on the critical defects includes: the number of each critical defect and the processing status of each critical defect; The determination of the degree of incompleteness of the functional critical defects in the target application based on the information of the critical defects specifically includes: Based on the processing status of each major defect, a corresponding processing weight value is set for each major defect; The degree of incompleteness of the functional critical defect is determined based on the processing weight value of each critical defect and the number of each critical defect; wherein, the processing weight value of each critical defect is added to the product of the number of each critical defect, and the sum is taken as the degree of incompleteness of the functional critical defect.

6. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Store information on the requirements and defects of the target application respectively; The correspondence between the requirements and defects is established using a relationship matrix.

7. A program evaluation system, characterized in that, include: The requirement unit is used to obtain requirement development information of the requirements involved in the target application, and to determine the degree of incompleteness of the requirements of the target application based on the requirement development information; The legacy unit is used to acquire test information of the target application during testing, and to determine the degree of legacy incompleteness of the target application based on the test information; wherein, the degree of legacy incompleteness is used to indicate the extent to which the discovered defects have not been addressed after the target application has been developed and tested, based on the test results. A critical defect unit is used to acquire information on critical defects in the target application and to determine the degree of incompleteness of the critical defects in the target application based on the critical defect information; the critical defects are defects with a preset defect level. The overall assessment unit is used to determine the overall incompleteness of the target application based on the incompleteness of the requirements, the incompleteness of the legacy applications, and the incompleteness of the major functional defects, so as to perform progress analysis on the target application. The test information of the target application during testing includes: information on test cases for multiple groups involved in testing the target application, the number of defects found, and the number of defects processed; The legacy unit specifically determines the classification test completion rate of each group based on the completion rate and proportion of test cases in each group; wherein, the classification test completion rate of any group is the product of the completion rate of test cases and the proportion of test cases in that group. The sum of the classification test completion rates of each group is taken as the overall test completion rate; Based on the overall test completion rate, the number of discovered defects, and the number of processed defects, the number of remaining defects in the target application is determined; wherein, the number of remaining defects includes the value obtained by subtracting the number of processed defects from the ratio of the number of discovered defects to the overall test completion rate; Set a first legacy weight value based on the processing status and a second legacy weight value based on the defect level for the legacy defects corresponding to the test cases; The degree of incompleteness of the target application is determined based on the number of remaining defects, the first remaining weight value, and the second remaining weight value; wherein the degree of incompleteness includes the product of the number of remaining defects, the first remaining weight value, and the second remaining weight value.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of computer programs adapted to be loaded by a processor and executed by the program evaluation method as described in any one of claims 1 to 6.

9. A terminal device, characterized in that, Including processor and memory; The memory is used to store a plurality of computer programs, the computer programs being loaded by a processor and executed by the program evaluation method as described in any one of claims 1 to 6; the processor is used to implement each of the plurality of computer programs.

Images