Code evaluation method, apparatus, device, and medium

Abstract

The present disclosure provides a code evaluation method, which can be applied to the field of information security and testing technology. The method comprises: obtaining change information, level information and developer information of a code to be evaluated; determining a total risk weight of the code to be evaluated according to the change information, the level information and the developer information; determining an evaluation pass threshold of the code to be evaluated according to the total risk weight; and determining a target evaluation result of the code to be evaluated according to the evaluation data of an evaluator obtained and the evaluation pass threshold of the code to be evaluated. The present disclosure also provides a code evaluation device, equipment, storage medium and program product.

Description

Technical Field

[0001] This disclosure relates to the fields of information security and testing, and more specifically to a code evaluation method, apparatus, electronic device, medium, and program product. Background Technology

[0002] Before going live and before a version update, application software needs to undergo code evaluation. Code evaluation is the process of checking whether the code conforms to coding standards and business requirements.

[0003] In related technologies, code evaluation processes often employ single or simple evaluation methods. However, in actual production environments, the code to be evaluated is influenced by various factors, such as the developers' capabilities. Because the actual circumstances of code submissions are not considered, relying solely on a single or simple evaluation method can lead to inconsistencies in the quality of the evaluated code, resulting in significant time and manpower costs for subsequent application maintenance. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a code evaluation method, apparatus, device, medium and program product.

[0005] According to a first aspect of this disclosure, a code evaluation method is provided, comprising: obtaining change information, grade information, and developer information of the code to be evaluated; determining the total risk weight of the code to be evaluated based on the change information, grade information, and developer information; determining an evaluation pass threshold for the code to be evaluated based on the total risk weight; and determining a target evaluation result for the code to be evaluated based on the obtained evaluation data of the evaluators and the evaluation pass threshold for the code to be evaluated.

[0006] According to embodiments of this disclosure, determining the total risk weight of the code to be evaluated based on change information, level information, and developer information includes: obtaining a first risk weight corresponding to the change information and a second risk weight corresponding to the level information, respectively; obtaining a third risk weight corresponding to the developer information; and determining the total risk weight of the code to be evaluated based on the first risk weight, the second risk weight, and the third risk weight.

[0007] According to an embodiment of this disclosure, the change information includes X change procedure files and the number of change lines corresponding to each change procedure file, where X ≥ 1; based on the change information and the level information, a first risk weight corresponding to the change information and a second risk weight corresponding to the level information are obtained, including: determining a first sub-risk weight corresponding to the xth change procedure file based on the number of change lines, where X ≥ x ≥ 1; determining the level information corresponding to the xth change procedure file based on the identification information of the xth change procedure file; and determining the second sub-risk weight corresponding to the xth change procedure file based on the level information.

[0008] According to embodiments of this disclosure, determining the total risk weight of the code to be evaluated based on a first risk weight, a second risk weight, and a third risk weight includes: determining an intermediate weight corresponding to the xth change procedure file based on a first sub-risk weight and a second sub-risk weight corresponding to the xth change procedure file; weighting the x intermediate weights to obtain the total change weight; and determining the total risk weight of the code to be evaluated based on the total change weight and the third risk weight.

[0009] According to embodiments of this disclosure, the evaluation data of the evaluators includes intermediate evaluation results and evaluation scores. The evaluation result of the code to be evaluated is determined based on the obtained evaluation data of the evaluators and the evaluation pass threshold of the code to be evaluated. This includes: obtaining the evaluation score of the evaluators if the intermediate evaluation result is determined to be pass; obtaining the historical evaluation score of the code to be evaluated based on the identification information of the code to be evaluated, where the historical evaluation score includes the sum of the evaluation scores of other evaluators, including evaluators whose intermediate evaluation result for the code to be evaluated is pass; and determining the evaluation result of the code to be evaluated based on the historical evaluation score, the evaluation score, and the evaluation pass threshold.

[0010] According to embodiments of this disclosure, the evaluation result of the code to be evaluated is determined based on historical evaluation scores, evaluation scores, and evaluation pass thresholds, including: if the total score obtained by adding the evaluation score to the historical evaluation scores is greater than or equal to the evaluation pass threshold, the evaluation result of the code to be evaluated is determined to be passed, and the code to be evaluated is stored; if the total score is less than the evaluation pass threshold, the total score is used as the new historical evaluation score, and the next evaluator is determined, until the updated total score is greater than or equal to the evaluation pass threshold.

[0011] A second aspect of this disclosure provides a code evaluation apparatus, comprising: an acquisition module for acquiring change information, grade information, and developer information of the code to be evaluated; a first determination module for determining the total risk weight of the code to be evaluated based on the change information, grade information, and developer information; a second determination module for determining an evaluation pass threshold for the code to be evaluated based on the total risk weight; and an evaluation module for determining a target evaluation result for the code to be evaluated based on the acquired evaluation data of the evaluators and the evaluation pass threshold of the code to be evaluated.

[0012] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the code evaluation method described above.

[0013] A fourth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the code evaluation method described above.

[0014] The fifth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described code evaluation method. Attached Figure Description

[0015] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 The system architecture of the code evaluation method according to embodiments of this disclosure is illustrated schematically;

[0017] Figure 2 A flowchart illustrating a code evaluation method according to an embodiment of the present disclosure is shown schematically;

[0018] Figure 3 A flowchart illustrating a method for determining total risk weights according to embodiments of the present disclosure is shown schematically.

[0019] Figure 4 A flowchart illustrating a code evaluation method according to a specific embodiment of the present disclosure is shown schematically;

[0020] Figure 5 A schematic block diagram of a code evaluation apparatus according to an embodiment of the present disclosure is shown; and

[0021] Figure 6 A block diagram of an electronic device suitable for a code evaluation method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0022] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0025] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0026] Currently, code evaluation systems often fail to differentiate between the specific circumstances of submitted code, applying a uniform or simplistic evaluation process. For example, whether modifying one or a hundred lines of code, or one line or a thousand lines, a single evaluator is sufficient for approval. However, in actual software development, the risk level of submitted code varies due to factors such as the number of lines changed, the importance of the changes, and the coding skills of the coders. Code with different risk levels should have different evaluation thresholds and be subject to varying degrees of control and handling.

[0027] Embodiments of this disclosure provide a code evaluation method, including: obtaining change information, level information, and developer information of the code to be evaluated; determining the total risk weight of the code to be evaluated based on the change information, level information, and developer information; determining the evaluation pass threshold of the code to be evaluated based on the total risk weight; and determining the target evaluation result of the code to be evaluated based on the obtained evaluation data of the evaluators and the evaluation pass threshold of the code to be evaluated.

[0028] Figure 1 The system architecture of the code evaluation method according to an embodiment of this disclosure is illustrated schematically.

[0029] like Figure 1 As shown, application scenario 100 according to this embodiment may include terminal devices 101, 102, and 103, network 104, and server 105. Network 104 is used as a medium to provide a communication link between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0030] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. For example, users can use terminal devices 101, 102, and 103 to complete code evaluation and interact with server 105 via network 104 to send evaluation results.

[0031] The code to be evaluated can also be obtained through terminal devices 101, 102, and 103, and interact with server 105 through network 104 to receive or send messages. The code evaluation is then completed through server 105.

[0032] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0033] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0034] It should be noted that the code evaluation method provided in this disclosure embodiment can generally be executed by server 105. Correspondingly, the code evaluation apparatus provided in this disclosure embodiment can generally be located in server 105. The code evaluation method provided in this disclosure embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the code evaluation apparatus provided in this disclosure embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0035] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0036] The following will be based on Figure 1 The described scene, through Figures 2-4 The code evaluation method of the disclosed embodiments is described in detail.

[0037] Figure 2 A flowchart illustrating a code evaluation method according to an embodiment of this disclosure is shown schematically.

[0038] like Figure 2 As shown, the method includes operations S210 to S240.

[0039] In operation S210, obtain change information, grade information, and developer information of the code to be evaluated.

[0040] According to embodiments of this disclosure, the change information includes whether the code to be evaluated has been changed, the number of lines changed, and the number of lines changed. The level information of the code to be evaluated is used to characterize the importance of the code. For example, the level information of code used to implement system functions, code used to implement application functions, and code used to implement interaction are different. For example, the higher the importance of the code, the higher the corresponding security level, and the higher the evaluation standard.

[0041] The code to be evaluated was developed by developers. Because developers have varying abilities, the quality of the code to be evaluated is related to the developer who submitted it. Developer information is used to determine the pass threshold for the evaluation of the code. Developer information includes developer identification information, developer level information, and developer risk level.

[0042] According to embodiments of this disclosure, change information, rating information, and developer information can all be directly obtained from a database. Specifically, change information and rating information can be obtained from the change list of the code to be evaluated in the database.

[0043] According to embodiments of this disclosure, after a developer submits code to be evaluated but before evaluation, the detection system can determine the change information of the code to be evaluated. For example, it can determine whether the code to be evaluated has been changed based on whether there is historical evaluation information. If there are historical evaluation results for the code to be evaluated, the historical evaluation code is compared with the code submitted this time to determine the change information of the code to be evaluated.

[0044] According to embodiments of this disclosure, the evaluation apparatus can also determine whether the code to be evaluated has changed.

[0045] In operation S220, the total risk weight of the code to be evaluated is determined based on change information, level information, and developer information.

[0046] According to embodiments of this disclosure, the risk level of the code to be evaluated is related to multiple factors, including change information, level information, and developer information. After obtaining the change information, level information, and developer information, the risk weight corresponding to each of the change information, level information, and developer information can be determined according to a pre-determined weight configuration table in the configuration library; the total risk weight of the code to be evaluated is determined by combining the risk weights of the change information, level information, and developer information.

[0047] According to embodiments of this disclosure, the total risk weight of the code to be evaluated can be determined based on the risk weights of change information, rating information, and developer information, and can also be determined based on other additional factors.

[0048] For example, determine the number of changes to the code to be evaluated and the corresponding risk weights; determine the number of vulnerabilities in the code to be evaluated and the corresponding risk weights. Combine multiple risk weights and calculate a weighted average to determine the total risk weight.

[0049] In operation S230, the threshold for passing the evaluation of the code to be evaluated is determined based on the total risk weight.

[0050] According to embodiments of this disclosure, the total risk weight can characterize multiple risk factors of the code to be evaluated, and multiple levels of total risk weight correspond to multiple evaluation criteria. Specifically, after determining the total risk weight, the score corresponding to the total risk weight is determined from the configuration library, and this score is used as the evaluation pass threshold for the code to be evaluated, i.e., the evaluation criterion.

[0051] According to embodiments of this disclosure, the pass threshold for the evaluation of the code to be evaluated is positively correlated with the total risk weight. That is, the higher the total risk weight, the higher the risk level of the code to be evaluated, and the higher the corresponding pass threshold.

[0052] In operation S240, the target evaluation result of the code to be evaluated is determined based on the evaluation data obtained from the evaluators and the evaluation pass threshold of the code to be evaluated.

[0053] According to embodiments of this disclosure, after determining the pass threshold for the code to be evaluated, the evaluation device can assign an evaluator to evaluate the code. After the evaluation, evaluation data is obtained, including intermediate evaluation results and evaluation scores. After the evaluator completes the evaluation, the evaluator's evaluation data is stored in a database.

[0054] According to embodiments of this disclosure, each evaluator's assessment score is a fixed value determined based on their assessment capabilities. Since the assessment capabilities of multiple evaluators vary, their assessment scores also differ. For example, evaluator level information can be determined based on their historical assessment information, with multiple levels corresponding to multiple fixed values. Alternatively, an evaluator's assessment score can be determined based on their error rate.

[0055] According to the embodiments disclosed herein, after the evaluators assigned by the evaluation system complete the evaluation, the evaluation data of the evaluators is obtained from the database, and the evaluation pass threshold of the code to be evaluated is determined from the configuration library. Based on the evaluation data and the evaluation pass threshold, the target evaluation result of the code to be evaluated can be determined.

[0056] Specifically, the evaluation data includes the interim evaluation results and scores from the evaluators. After acquiring the evaluation data, if the interim evaluation result is "fail," the evaluation device determines the target evaluation result for the code to be evaluated as "fail." If the interim evaluation result is "pass," the target evaluation result is determined based on the evaluator's score and the pass threshold for the code to be evaluated.

[0057] According to embodiments of this disclosure, the evaluation device can assign one or more evaluators to conduct evaluations based on the evaluator's evaluation score and the evaluation pass threshold of the code to be evaluated, in order to determine the target evaluation result.

[0058] For example, if evaluator A's evaluation ability is higher than evaluator B's, then evaluator A's evaluation score will be higher than evaluator B's. For the same evaluation code, if evaluator A's score alone reaches the evaluation threshold, the target evaluation result is determined as passed. However, if evaluator B's score alone does not reach the evaluation threshold, the evaluation score of evaluator C needs to be added to achieve the passing threshold. The evaluation device can assign evaluator C or other evaluators to continue the evaluation after evaluator B completes their assessment. This ensures both evaluation quality and improves evaluation efficiency.

[0059] This disclosure considers factors that affect code during the code evaluation process. It determines the total risk weight by obtaining change information, risk level information, and developer information for the code to be evaluated. Then, it determines the evaluation pass threshold based on the total risk weight. This allows for determining the evaluation pass threshold based on the actual situation of the code being evaluated, improving the flexibility of the code evaluation process. Since the evaluation pass threshold is directly proportional to the total risk weight, the quality of the code being evaluated is ensured while flexibly determining its risk level. Determining the target evaluation result through the evaluators' evaluation data and the evaluation pass threshold ensures both the quality and efficiency of the code evaluation process while flexibly determining the evaluation pass threshold.

[0060] Ensuring the quality of code evaluation can reduce the amount of obsolete code and the waste of storage resources. Furthermore, by guaranteeing the quality of the code evaluation process, it is possible to reduce the amount of flawed code resulting from simplistic evaluation methods in actual production. This, in turn, reduces the time developers spend reworking code, the time spent updating and maintaining applications, lowers development costs, and improves development efficiency.

[0061] Figure 3 A flowchart illustrating a method for determining total risk weights according to an embodiment of this disclosure is shown schematically.

[0062] like Figure 3 As shown, the determination of the total risk weight in this embodiment includes operations S321 to S323, which can be used as a specific embodiment of operation S220.

[0063] In operation S321, based on the change information and the level information, the first risk weight corresponding to the change information and the second risk weight corresponding to the level information are obtained respectively.

[0064] In operation S322, based on the developer information, obtain the third risk weight corresponding to the developer information.

[0065] In operation S323, the total risk weight of the code to be evaluated is determined based on the first risk weight, the second risk weight, and the third risk weight.

[0066] According to embodiments of this disclosure, the change information includes the number of lines of code to be evaluated that have been changed, with different numbers of lines of change corresponding to different risk levels. The weight configuration table in the configuration library includes a first risk weight corresponding to different numbers of lines of change. Code changes include at least one of additions, deletions, and modifications.

[0067] According to embodiments of this disclosure, the number of changed rows within a certain range can correspond to a first risk weight. The first risk weight corresponding to the range of changed row numbers is denoted as L, where L(a) represents the first risk weight corresponding to the range of changed row numbers a. The first risk weight L is proportional to the change function.

[0068] For example, the interval 1 is [1, 50), and the corresponding first risk weight L(1) = 1; the interval 2 is [50, 100), and the corresponding first risk weight L(2) = 1.1.

[0069] The risk level information includes the importance of the code, with different business functions having corresponding risk weights. A second risk weight corresponding to the risk level information can be determined based on the weight configuration table in the configuration library. Let M represent the second risk weight corresponding to the code, where M(b) represents the second risk weight corresponding to the importance of code b. The importance of the code is directly proportional to the second risk weight; the more important the code, the greater the second risk weight.

[0070] For example, code A is a level 3 ordinary code with a second risk weight M(A) = 1; program B is a level 2 core program with a second risk weight M(B) = 1.2; and program C is a level 1 core security program with a second risk weight M(C) = 1.5.

[0071] Developer information includes the risk level of the code written by the developer. The third risk weight corresponding to the developer's information can be determined based on the weight configuration table in the configuration library. Let N represent the third risk weight corresponding to a developer, where N(c) represents the third risk weight of the code developed by developer c. The more experience and coding ability a developer has, the lower the risk of the code modified by that developer, and therefore the lower the corresponding risk weight; conversely, the less experience and the less coding ability a developer has, the higher the risk weight.

[0072] For example, developer 1 has strong coding ability, and the corresponding third risk weight N(1) = 1; developer 2 has medium coding ability, and the corresponding third risk weight N(2) = 1.2; developer 3 has weak coding ability, and the corresponding third risk weight N(1) = 1.5.

[0073] The specific weight values ​​corresponding to the first risk weight, the second risk weight, and the third risk weight can be determined from the weight configuration table in the configuration library.

[0074] According to embodiments of this disclosure, the total risk weight is denoted as W, which is determined based on a first risk weight L, a second risk weight M, and a third risk weight N. For example, W = L * M * N.

[0075] According to embodiments of this disclosure, after determining the total risk weight and determining the corresponding evaluation pass threshold from the configuration library based on the total risk weight, the target evaluation result of the code to be evaluated can be determined based on the evaluation data of the evaluator and the evaluation pass threshold.

[0076] The assessment data includes intermediate assessment results and assessment scores. The assessment score is denoted as O, where O(d) represents the score at which assessment d passes. The assessment pass threshold is denoted as P, and it is determined based on the total risk weight. Specifically, different ranges of total risk weight W correspond to different assessment pass thresholds. P(e) represents the assessment pass threshold required to pass the assessment corresponding to the total risk weight range e.

[0077] Interim evaluation results include pass and fail. If the interim evaluation result is determined to be fail, the target evaluation result for the code to be evaluated can be directly determined to be fail. If the interim evaluation result is pass, it is determined whether the evaluation score of the evaluator reaches the evaluation pass threshold.

[0078] According to embodiments of this disclosure, when an intermediate evaluation result is determined to be unsuccessful, the evaluation score of the evaluator is retrieved from the configuration library. Since other evaluators may have already evaluated the code to be evaluated, and the intermediate evaluation result was successful, but the evaluation pass threshold was not met, the historical evaluation scores of the code to be evaluated can be retrieved from the database based on the identification information of the code to be evaluated.

[0079] After obtaining the historical evaluation scores of the code to be evaluated and the evaluation scores of the evaluators, it can be determined whether the total score has reached the evaluation pass threshold.

[0080] According to embodiments of this disclosure, if the total score obtained by adding the evaluation score and the historical evaluation scores is greater than or equal to the evaluation pass threshold, the evaluation result of the code to be evaluated is determined to be passed, and the code to be evaluated is stored.

[0081] If the total score is less than the assessment pass threshold, the total score will be used as the new historical assessment score, and the next assessor will be determined until the updated total score is greater than or equal to the assessment pass threshold.

[0082] For example, if the code to be evaluated has not been evaluated by an evaluator, and evaluator 1 is the first evaluator, the corresponding total score is T1, the historical evaluation score is H1, and the evaluation score of evaluator 1 is O(1), then the total score T1 = H1 + O(1), and the historical evaluation score H1 is 0.

[0083] If the total score T1 ≥ P, the target evaluation result of the code to be evaluated is determined to be pass, and the code to be evaluated is stored.

[0084] When the total score T1 < P, the evaluation device assigns evaluator 2 to evaluate the code to be evaluated. At this time, the historical evaluation score H2 is updated to the total score T1, i.e., H2 = T1 = O(1). If the intermediate evaluation result of evaluator 2 is "pass", the total score of the code to be evaluated at this time is T2 = H2 + O(2) = O(1) + O(2). If T2 ≥ P, the evaluation result of the code to be evaluated is determined to be "pass", and the code to be evaluated is stored. If T2 < P, the evaluation device continues to assign evaluator 3 to evaluate the code to be evaluated until the updated total score is greater than or equal to the evaluation pass threshold.

[0085] According to embodiments of this disclosure, the code to be evaluated may include a modified procedure file, corresponding to a first risk weight, a second risk weight, a third risk weight, and a final total risk weight. The code to be evaluated may also include X modified procedure files, corresponding to the first risk weight comprising X first sub-risk weights, X second sub-risk weights, and a third risk weight corresponding to the developer.

[0086] According to embodiments of this disclosure, the change information includes X change procedure files and the number of change lines corresponding to each change procedure file, where X ≥ 1. Based on the number of change lines in the change information, a first sub-risk weight corresponding to the x-th change procedure file is determined, where X ≥ x ≥ 1. Based on the identification information of the x-th change procedure file, level information corresponding to the x-th change procedure file is determined; based on the level information, a second sub-risk weight corresponding to the x-th change procedure file is determined.

[0087] Similarly, we obtain X first sub-risk weights and X second sub-risk weights corresponding to each of the X changed procedure files. Since the third risk weight is related to the developer, when the code to be evaluated includes X changed procedure files, as long as the developer has not changed, there is a unique corresponding third risk weight.

[0088] For X modified procedure files, based on the modification information and level information corresponding to each modified procedure file, the first sub-risk weight and the second sub-risk weight of each modified procedure file can be determined. The first sub-risk weight of the xth sub-code is L(Y(x)), which represents the first sub-risk weight determined based on the number of modified lines Y(x) of the xth modified procedure file; the second sub-risk weight is M(x).

[0089] According to an embodiment of this disclosure, after determining the first sub-risk weight and the second sub-risk weight corresponding to the xth modified procedure file, the intermediate weight corresponding to the xth modified procedure file can be obtained based on the first sub-risk weight and the second sub-risk weight.

[0090] Similarly, we obtained the X intermediate weights corresponding to the X modified program files that were changed.

[0091] Specifically, preferably, the intermediate weight can be the product of the first sub-risk weight and the second sub-risk weight. Depending on the implementation, the intermediate weight can also be a weighted sum of the first and second sub-risk weights.

[0092] After determining the X intermediate weights corresponding to the X change procedure files, the total risk weight W of the code to be evaluated can be determined based on the X intermediate weights and the third risk weight N corresponding to the developer information.

[0093] According to embodiments of this disclosure, determining the total risk weight based on X intermediate weights and a third risk weight includes: weighting and summing the X intermediate weights to obtain the total change weight; and determining the total risk weight of the code to be evaluated based on the total change weight and the third risk weight.

[0094] For the code to be evaluated, there are X change procedure files, i.e., the change procedure list for the code to be evaluated is X(1, 2, ..., i, ..., X) containing X change procedure files. For ease of description, the i-th change procedure file in the change procedure list is denoted as X(i). The total risk weight W of the code to be evaluated satisfies:

[0095]

[0096] Wherein, N(c) represents the third risk weight of assessor c, Y(i) represents the range of changed lines in the i-th change procedure file, L(Y(i)) represents the first risk weight corresponding to the range of changed lines Y(i); X(i) represents the i-th sub-code that has been changed in the change list, and M(X(i)) represents the second risk weight corresponding to the i-th sub-code. X indicates that a total of X sub-codes have been changed.

[0097] Table 1 Configuration Table

[0098]

[0099] As shown in Table 1, from left to right, the table represents the first risk weight L, the second risk weight M, the third risk weight N, the assessment pass threshold P, and the assessor's assessment score O. The first risk weight L corresponds to the number of changed rows, and includes two columns representing the range of changed rows and the first risk weight corresponding to different ranges. The second risk weight M corresponds to the code level, and includes two columns representing the code identifier and the second risk weight corresponding to different code levels. The third risk weight N corresponds to the developer's development capability, and includes two columns representing the developer identifier and the third risk weight corresponding to different developers. The assessment pass threshold P includes the total risk weight range and the assessment pass threshold corresponding to different total risk weight ranges. The assessor's assessment score O includes the assessor identifier and the assessment score corresponding to different assessors.

[0100] For example, if developer 1 submits code that modifies 80 lines of code B, with a first risk weight of 1.1, a second risk weight of 1.2, and a third risk weight of 1, then the total risk weight W of the submitted code is 1 * 1.2 * 1.1 = 1.32, and the corresponding pass threshold for evaluation is 2.

[0101] If evaluators 2 or 3 have intermediate evaluation scores of 2 or 3 respectively, both higher than 2, then the target evaluation result for the submitted code is "pass" if either evaluator 2 or 3 has an intermediate evaluation score of 1. However, evaluator 1 has an evaluation score of 1, lower than 2. Even with an intermediate evaluation score of 1, this score is below the passing threshold, requiring other evaluators to continue the evaluation. Only when the intermediate evaluation scores of other evaluators are "pass," and the sum of the evaluation scores of evaluator 1 and the other evaluators is greater than 2, will the target evaluation result for the submitted code be "pass."

[0102] For example, if developer 2 submits code for evaluation that has modified 30 lines of code A and 80 lines of code B, with the first sub-risk weight of code A being 1 and the second sub-risk weight being 1, and the first sub-risk weight of code B being 1.1, the second sub-risk weight being 1.2, and the third sub-risk weight being 1.2, then the total risk weight W of the code submitted for evaluation is 1.2*(1*1+1.2*1.1)=2.784, corresponding to an evaluation pass threshold of 3.

[0103] The evaluation score for evaluator 3 is 3. Therefore, if the intermediate evaluation result for evaluator 3 is "pass", the evaluation score is greater than the evaluation pass threshold, and the target evaluation result for the code to be evaluated is "pass".

[0104] If evaluator 1's intermediate evaluation result is "pass," but their evaluation score is less than the passing threshold, evaluator 2 needs to be assigned to conduct the evaluation. If evaluator 2 conducts the evaluation, the historical passing threshold is evaluator 1's evaluation score. If evaluator 2's intermediate evaluation result is also "pass," and the sum of their historical evaluation score and evaluator 2's evaluation score exceeds the passing threshold, then the target evaluation result for this submitted code is "pass."

[0105] Figure 4 A flowchart illustrating a code evaluation method according to a specific embodiment of this disclosure is shown.

[0106] like Figure 4 As shown, in operation S401, after the developer submits the code to be evaluated to the evaluation system, the evaluation begins. Operation S402 retrieves the change information, level information, and developer information of the code to be evaluated. After retrieving the change information, level information, and developer information, operation S403 retrieves multiple risk weights from the configuration library based on the above information. Specifically, these correspond to a first risk weight, a second risk weight, and a third risk weight. Then, operation S404, after retrieving multiple risk weights, calculates the total risk weight of the code to be evaluated by combining these multiple risk weights. Operation S405 retrieves the evaluation pass threshold corresponding to the total risk weight from the configuration library based on the total risk weight.

[0107] In operation S406, the evaluation device identifies the evaluator for the code to be evaluated, and then proceeds to operation S407. After the evaluator performs the evaluation, evaluation data is obtained. The evaluation data includes intermediate evaluation results and evaluation scores. In operation S408, it is determined whether the evaluator's intermediate evaluation result is pass. If the intermediate evaluation result is determined to be fail, operation S414 is initiated, the target evaluation result for the code to be evaluated is set to fail, and operation S415 is initiated to end the evaluation.

[0108] If the intermediate evaluation result is determined to be passed, proceed to operation S409 to obtain the historical evaluation score of the code to be evaluated, and then proceed to operation S410 to calculate the total score of the evaluator's evaluation score and the historical evaluation score.

[0109] In operation S411, determine whether the total score is greater than or equal to the evaluation pass threshold. If the total score is greater than or equal to the evaluation pass threshold, proceed to operation S412, and set the target evaluation result of the code to be evaluated to pass. In operation S413, integrate the code to be evaluated into the code library, and proceed to operation S415 to end the evaluation.

[0110] If the total score is less than the assessment pass threshold, return to operation S406 to re-determine the assessor until the accumulated total score is greater than or equal to the assessment pass threshold.

[0111] Figure 5 A schematic block diagram of a code evaluation apparatus according to an embodiment of the present disclosure is shown.

[0112] like Figure 5 As shown, the code evaluation device 500 of this embodiment includes an acquisition module 510, a first determination module 520, a second determination module 530, and an evaluation module 540.

[0113] The acquisition module 510 is used to acquire change information, level information, and developer information of the code to be evaluated. In one embodiment, the acquisition module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0114] The first determining module 520 is used to determine the total risk weight of the code to be evaluated based on change information, level information, and developer information. In one embodiment, the first determining module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0115] The second determining module 530 is used to determine the evaluation pass threshold of the code to be evaluated based on the total risk weight. In one embodiment, the second determining module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0116] The evaluation module 540 is used to determine the target evaluation result of the code to be evaluated based on the evaluation data obtained from the evaluators and the evaluation pass threshold of the code to be evaluated. In one embodiment, the evaluation module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0117] According to embodiments of this disclosure, the second determining module 530 includes a first determining unit, a second determining unit, and a third determining unit.

[0118] The first determining unit is used to obtain a first risk weight corresponding to the change information and a second risk weight corresponding to the level information, respectively, based on the change information and the level information. In one embodiment, the first determining unit can be used to perform the operation S321 described above, which will not be repeated here.

[0119] The second determining unit is used to obtain a third risk weight corresponding to the developer information based on the developer information. In one embodiment, the second determining unit can be used to perform the operation S322 described above, which will not be repeated here.

[0120] The third determining unit is used to determine the total risk weight of the code to be evaluated based on the first risk weight, the second risk weight, and the third risk weight. In one embodiment, the second determining unit can be used to perform the operation S323 described above, which will not be repeated here.

[0121] Figure 6 A block diagram of an electronic device suitable for a code evaluation method according to an embodiment of the present disclosure is shown schematically.

[0122] like Figure 6 As shown, an electronic device 600 according to an embodiment of this disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0123] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0124] According to embodiments of this disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0125] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0126] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0127] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the code evaluation method provided in the embodiments of this disclosure.

[0128] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0129] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0130] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0131] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0133] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0134] The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this disclosure. It should be understood that the above descriptions are merely specific embodiments of this disclosure and are not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A code evaluation method, comprising: Obtain change information, rating information, and developer information of the code to be evaluated; The total risk weight of the code to be evaluated is determined based on the change information, the level information, and the developer information. Based on the total risk weight, determine the evaluation pass threshold for the code to be evaluated; as well as Based on the assessment data obtained from the assessors and the assessment pass threshold of the code to be assessed, the target assessment result of the code to be assessed is determined. The assessment data from the assessors includes intermediate assessment results and assessment scores. Based on the assessment data obtained from the assessors and the assessment pass threshold of the code to be assessed, the assessment result of the code to be assessed is determined, including: If the intermediate evaluation result is determined to be passing, the evaluation score of the evaluator is obtained; Based on the identification information of the code to be evaluated, the historical evaluation score of the code to be evaluated is obtained. The historical evaluation score includes the sum of the evaluation scores of other evaluators, including evaluators whose intermediate evaluation result for the code to be evaluated was "pass". If the total score obtained by adding the evaluation score to the historical evaluation score is greater than or equal to the evaluation pass threshold, the evaluation result of the code to be evaluated is determined to be passed, and the code to be evaluated is stored. If the total score is less than the assessment pass threshold, the total score is used as the new historical assessment score, and the next assessor is determined, until the updated total score is greater than or equal to the assessment pass threshold.

2. The method according to claim 1, wherein, Based on the change information, the rating information, and the developer information, the total risk weight of the code to be evaluated is determined as follows: Based on the change information and the level information, a first risk weight corresponding to the change information and a second risk weight corresponding to the level information are obtained respectively. Based on the developer information, obtain the third risk weight corresponding to the developer information; and The total risk weight of the code to be evaluated is determined based on the first risk weight, the second risk weight, and the third risk weight.

3. The method according to claim 2, wherein, The change information includes X change procedure files and the number of change lines corresponding to each change procedure file, where X ≥ 1; Based on the change information and the level information, a first risk weight corresponding to the change information and a second risk weight corresponding to the level information are obtained, including: Based on the number of changed lines, determine the first sub-risk weight corresponding to the x-th changed procedure file, where X ≥ x ≥ 1; Based on the identification information of the xth change procedure file, determine the level information corresponding to the xth change procedure file; Based on the aforementioned level information, determine the second sub-risk weight corresponding to the xth change procedure file.

4. The method according to claim 3, wherein, The total risk weight of the code to be evaluated is determined based on the first risk weight, the second risk weight, and the third risk weight, including: Based on the first sub-risk weight and the second sub-risk weight corresponding to the xth change procedure file, determine the intermediate weight corresponding to the xth change procedure file; Sum the X intermediate weights to obtain the total change weight; and The total risk weight of the code to be evaluated is determined based on the total change weight and the third risk weight.

5. A code evaluation apparatus, comprising: The acquisition module is used to acquire change information, level information, and developer information of the code to be evaluated; The first determining module is used to determine the total risk weight of the code to be evaluated based on the change information, the level information, and the developer information. The second determining module is used to determine the evaluation pass threshold of the code to be evaluated based on the total risk weight; as well as The evaluation module is used to determine the target evaluation result of the code to be evaluated based on the evaluation data obtained from the evaluators and the evaluation pass threshold of the code to be evaluated; the evaluation data from the evaluators includes intermediate evaluation results and evaluation scores; The evaluation module is also used to obtain the evaluation score of the evaluator when the intermediate evaluation result is determined to be passed; Based on the identification information of the code to be evaluated, the historical evaluation score of the code to be evaluated is obtained. The historical evaluation score includes the sum of the evaluation scores of other evaluators, including evaluators whose intermediate evaluation result for the code to be evaluated was "pass". If the total score obtained by adding the evaluation score to the historical evaluation score is greater than or equal to the evaluation pass threshold, the evaluation result of the code to be evaluated is determined to be passed, and the code to be evaluated is stored. If the total score is less than the assessment pass threshold, the total score is used as the new historical assessment score, and the next assessor is determined, until the updated total score is greater than or equal to the assessment pass threshold.

6. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 4.

Images