Code quality evaluation method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of code evaluation, in particular to a code quality evaluation method and device, computer equipment and a storage medium. The method comprises the steps of obtaining test code data output by a to-be-detected model and a single-side operation result after single-side operation of the test code data; performing code quality evaluation on the to-be-detected model according to the test code data and / or the single-side operation result to respectively obtain a modification ratio score, an effectiveness score, a time efficiency score, a diversity score and a coverage rate score corresponding to the to-be-detected model; and determining a code quality score of the to-be-detected model according to the modification ratio score, the validity score, the time efficiency score, the diversity score and the coverage rate score. According to the method, comprehensive detection and evaluation of the test code data generated for the to-be-detected model are realized, operation and maintenance personnel do not need to have a relatively high professional knowledge level, and the quality evaluation efficiency of the test code data output by the to-be-detected model is ensured.

Description

Technical Field

[0001] This application relates to the technical field of code evaluation, and particularly to a code quality evaluation method, apparatus, computer device, and storage medium. Background Art

[0002] With the rapid development of large language models in the fields of natural language processing and code generation, their applications in software development have gradually attracted attention. Unit testing, as a key link in software development, focuses on verifying the basic units of software to ensure the correctness and reliability of the code. In recent years, large language models have demonstrated great potential in the field of code generation with their excellent code understanding capabilities.

[0003] However, in the prior art, in order to evaluate the code generation quality of large models in unit test cases, it is necessary to evaluate through manual discrimination. This process not only has certain requirements for the professional knowledge level of evaluators, but also affects the code quality evaluation efficiency and cannot quickly obtain the code quality evaluation results. Summary of the Invention

[0004] Based on this, in order to solve the above technical problems, it is necessary to provide a code quality evaluation method, apparatus, computer device, and storage medium that can quickly obtain code quality evaluation results.

[0005] In a first aspect, this application provides a code quality evaluation method. The method includes:

[0006] Obtain the test code data output by the model to be detected, and the single-sided operation result after the single-sided operation of the test code data;

[0007] Perform code quality evaluation on the model to be detected according to the test code data and / or the single-sided operation result, and obtain the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage rate score corresponding to the model to be detected respectively;

[0008] Determine the code quality score of the model to be detected according to the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage rate score.

[0009] In one of the embodiments, performing code quality evaluation on the model to be detected according to the test code data to obtain the modification ratio score corresponding to the model to be detected includes:

[0010] Extract the abnormal code data included in the test code data;

[0011] Determine the modification ratio score corresponding to the model to be detected according to the number of abnormal characters corresponding to the abnormal code data and the total number of characters of the test code data.

[0012] In one embodiment, the code quality of the model to be detected is evaluated based on the unilateral running result, and the effectiveness score corresponding to the model to be detected is obtained, including:

[0013] Obtain the standard defect code segments included in the preset benchmark test code dataset and the total amount of code corresponding to the standard defect code segments;

[0014] Determine the defect code text description included in the unilateral running result;

[0015] Select similar subcodes with a first similarity greater than a first threshold between the defect subcode text descriptions included in the defect code text description and the standard defect code segments;

[0016] Determine the effectiveness score corresponding to the model to be detected according to the number of codes of the similar subcodes and the total amount of code.

[0017] In one embodiment, the code quality of the model to be detected is evaluated based on the unilateral running result, and the time efficiency score corresponding to the model to be detected is obtained, including:

[0018] Determine the average unilateral running time of the test code data according to the total execution time corresponding to the unilateral running result and the number of unilateral runs;

[0019] Determine the time efficiency score corresponding to the model to be detected according to the average unilateral running time and the preset time threshold.

[0020] In one embodiment, the code quality of the model to be detected is evaluated based on the test code data, and the diversity score corresponding to the model to be detected is obtained, including:

[0021] Obtain the standard diversity scenario data included in the preset benchmark test code dataset and the total amount of scenarios corresponding to the standard diversity scenario data;

[0022] Determine the test scenario data corresponding to the test code data;

[0023] Select similar sub-scenario data with a second similarity greater than a second threshold between the sub-scenario data included in the test scenario data and the standard diversity scenario data;

[0024] Determine the diversity score corresponding to the model to be detected according to the number of similarities of the similar sub-scenario data and the total amount of scenarios.

[0025] In one embodiment, the code quality of the model to be detected is evaluated based on the unilateral running result, and the coverage score corresponding to the model to be detected is obtained, including:

[0026] Extract the coverage value from the coverage report corresponding to the unilateral running result;

[0027] Determine the coverage rate score corresponding to the model to be detected according to the coverage rate value.

[0028] In one embodiment, according to the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage rate score, determine the code quality score of the model to be detected, including:

[0029] Determine the first weight corresponding to the modification ratio score, the second weight corresponding to the effectiveness score, the third weight corresponding to the time efficiency score, the fourth weight corresponding to the diversity score, and the fifth weight corresponding to the coverage rate score;

[0030] Perform weighted processing on the modification ratio score according to the first weight to obtain the first score;

[0031] Perform weighted processing on the effectiveness score according to the second weight to obtain the second score;

[0032] Perform weighted processing on the time efficiency score according to the third weight to obtain the third score; [[ID=IS]]

[0033] Perform weighted processing on the diversity score according to the fourth weight to obtain the fourth score;

[0034] Perform weighted processing on the coverage rate score according to the fifth weight to obtain the fifth score;

[0035] Perform a sum operation on the first score, the second score, the third score, the fourth score, and the fifth score to obtain the code quality score of the model to be detected.

[0036] In a second aspect, the present application also provides a code quality evaluation device. The device includes:

[0037] An acquisition module, configured to acquire the test code data output by the model to be detected, and the unilateral operation result after the test code data is unidirectionally run;

[0038] An evaluation module, configured to perform code quality evaluation on the model to be detected according to the test code data and / or the unilateral operation result, and respectively obtain the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage rate score corresponding to the model to be detected;

[0039] A determination module, configured to determine the code quality score of the model to be detected according to the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage rate score.

[0040] In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0041] Obtain the test code data output by the model to be detected, and the unilateral operation results after the test code data undergoes unilateral operation;

[0042] Conduct code quality assessment on the model to be detected according to the test code data and / or the unilateral operation results, and respectively obtain the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage score corresponding to the model to be detected;

[0043] Determine the code quality score of the model to be detected according to the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage score.

[0044] Fourthly, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0045] Obtain the test code data output by the model to be detected, and the unilateral operation results after the test code data undergoes unilateral operation;

[0046] Conduct code quality assessment on the model to be detected according to the test code data and / or the unilateral operation results, and respectively obtain the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage score corresponding to the model to be detected;

[0047] Determine the code quality score of the model to be detected according to the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage score.

[0048] Fifthly, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0049] Obtain the test code data output by the model to be detected, and the unilateral operation results after the test code data undergoes unilateral operation;

[0050] Conduct code quality assessment on the model to be detected according to the test code data and / or the unilateral operation results, and respectively obtain the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage score corresponding to the model to be detected;

[0051] Determine the code quality score of the model to be detected according to the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage score.

[0052] The above code quality evaluation method, device, computer device and storage medium realize the code quality evaluation of the model to be detected through the test code data output by the model to be detected and the unilateral operation results after the unilateral operation of the test code data, and obtain the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score corresponding to the model to be detected. Furthermore, according to the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score, the code quality score of the model to be detected is determined. According to the above content, when the present application needs to perform code quality evaluation on the model to be detected, it will pre-obtain the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score from five aspects: modification ratio, code effectiveness, time efficiency, scenario diversity and code coverage, so as to realize the comprehensive detection and evaluation of the test code data generated for the model to be detected, without the need for operation and maintenance personnel to have a high level of professional knowledge, and ensure the quality evaluation efficiency of the test code data output by the model to be detected. Description of the Drawings

[0053] Figure 1 It is an application environment diagram of a code quality evaluation method provided by an embodiment of the present application;

[0054] Figure 2 It is a flowchart of the first code quality evaluation method provided by an embodiment of the present application;

[0055] Figure 3 It is a flowchart of the second code quality evaluation method provided by an embodiment of the present application;

[0056] Figure 4 It is a flowchart of the third code quality evaluation method provided by an embodiment of the present application;

[0057] Figure 5 It is a flowchart of the fourth code quality evaluation method provided by an embodiment of the present application;

[0058] Figure 6 It is a flowchart of the fifth code quality evaluation method provided by an embodiment of the present application;

[0059] Figure 7 It is a flowchart of the sixth code quality evaluation method provided by an embodiment of the present application;

[0060] Figure 8 It is a flowchart of the seventh code quality evaluation method provided by an embodiment of the present application;

[0061] Figure 9 It is a structural block diagram of a code quality evaluation device provided by an embodiment of the present application;

[0062] Figure 10 The internal structure diagram of a computer device in an embodiment. Specific implementation manners

[0063] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] The code quality evaluation method provided by the embodiments of the present application can be applied to, for example, Figure 1 the application environment shown in the figure. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed in the cloud or other network servers. Through the test code data output by the model to be detected and the unilateral running result after the test code data is unilaterally run, the code quality evaluation of the model to be detected is realized, and the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score corresponding to the model to be detected are obtained. Furthermore, according to the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score, the code quality score of the model to be detected is determined. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0065] In one embodiment, as Figure 2 shown in the figure, a code quality evaluation method is provided. Taking the method applied to Figure 1 the server 104 in the figure as an example, the method includes the following steps:

[0066] S201, obtain the test code data output by the model to be detected and the unilateral running result after the test code data is unilaterally run.

[0067] Among them, the test code data refers to the code generated by the model to be detected; the unilateral running result refers to the result data obtained after running and testing the test code data in a unit test running sandbox environment.

[0068] In an embodiment of the present application, the method for constructing a unit test running sandbox environment includes: building an environment base through Docker (application container engine) containers, and installing a compilation and execution environment for the corresponding programming language on the environment base; installing the corresponding unit test framework dependency libraries and extensions on the environment base; installing the corresponding unit test framework code coverage tool on the environment base; furthermore, constructing a unit test executor, and prefabricating execution commands for the unit test frameworks of each programming language through the unit test executor to achieve the construction of the unit test running sandbox environment.

[0069] It should be noted that when it is necessary to obtain the test code data output by the model to be detected, the code under test for input to the model to be detected and the usage example code can be pre-constructed; furthermore, the code under test and the usage example code are input into the model to be detected to obtain the test code data output by the model to be detected.

[0070] In an embodiment of the present application, a benchmark test code dataset can be pre-constructed; furthermore, the code under test and the usage example code are read from the benchmark test code dataset; the code under test and the usage example code are concatenated to obtain a large model prompt statement; the large model prompt statement is input into the inference interface of the model to be detected to obtain the output result of the model to be detected; the output result is cleaned to obtain the test code data output by the model to be detected; the test code data is run in the unit test running sandbox environment, and the running data is recorded to obtain the unit test running result after the unit test of the test code data.

[0071] Among them, the data contained in the benchmark test code dataset is composed of fields such as "programming language (language)", "code under test" (code), "usage example code" (usageCode), "standard defect code segment" (bugSet), "defect mark" (bugFlag), and "standard diversity scenario data" (diversityScene). The above content is the data format of the benchmark test code dataset.

[0072] In an embodiment of the present application, when it is necessary to generate a benchmark test code dataset, it may include the following content: generating a benchmark test code dataset in an artificial production manner, so that the operation and maintenance personnel generate a benchmark test code dataset according to the data format of the data in the benchmark test code dataset.

[0073] In another embodiment of the present application, when it is necessary to generate a benchmark test code dataset, a pre-trained data generation model can also be obtained, and the data format of the data in the benchmark test code dataset is used as a data prompt, so as to input the data prompt and an example data corresponding to the data prompt into the data generation model, so that the data generation model generates data according to the data prompt to achieve the purpose of generating a benchmark test code dataset.

[0074] S202. Perform code quality assessment on the model to be detected according to the test code data and / or unilateral operation results, and obtain the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage rate score corresponding to the model to be detected respectively.

[0075] It should be noted that the modification ratio score is used to characterize the level of the modification ratio of the test code data. Specifically, the modification ratio refers to the ratio of the modified characters in the test code data to the total characters in the test code data after the test code data has flaws that cause it to fail to compile or execute and can be successfully compiled or executed after being modified by the operation and maintenance personnel.

[0076] The effectiveness score is used to characterize the level of the effectiveness of the test code data. Specifically, the level of effectiveness is used to characterize whether the test code data can accurately detect errors and defects in the code.

[0077] The time efficiency score is used to characterize the level of time efficiency during the execution of the test code data. Specifically, the time efficiency is used to characterize whether the test code data can be executed within a preset time period.

[0078] The diversity score is used to characterize the level of scenario diversity corresponding to the test code data. Specifically, the scenario diversity is used to characterize whether the test code can cover various different inputs, scenarios, boundary conditions, and abnormal situations.

[0079] The coverage rate score is used to characterize the level of the coverage rate of the code library during the execution of the test code data. Specifically, the coverage rate can effectively reflect whether the code data included in the test code data is fully tested.

[0080] S203. Determine the code quality score of the model to be detected according to the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage rate score.

[0081] In an embodiment of the present application, when it is necessary to determine the code quality score of the model to be detected, a sum operation can be performed on the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage rate score, and the obtained result is the code quality score of the model to be detected.

[0082] In another embodiment of the present application, when it is necessary to determine the code quality score of the model to be detected according to the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage rate score, the following content may be included:

[0083] Determine the first weight corresponding to the modification ratio score, the second weight corresponding to the effectiveness score, the third weight corresponding to the time efficiency score, the fourth weight corresponding to the diversity score, and the fifth weight corresponding to the coverage score; perform weighted processing on the modification ratio score according to the first weight to obtain the first score; perform weighted processing on the effectiveness score according to the second weight to obtain the second score; perform weighted processing on the time efficiency score according to the third weight to obtain the third score; perform weighted processing on the diversity score according to the fourth weight to obtain the fourth score; perform weighted processing on the coverage score according to the fifth weight to obtain the fifth score; perform a sum operation on the first score, the second score, the third score, the fourth score, and the fifth score to obtain the code quality score of the model to be detected.

[0084] Specifically, the total score corresponding to the code quality score is 100 points, and the score range of the code quality score is 1 - 100. The first weight is preset to 0.1, the second weight is 0.3, the third weight is 0.1, the fourth weight is 0.2, and the fifth weight is 0.3; in summary, the first weight to the fifth weight in this application can be set or adjusted according to the actual situation, and the value range of the first weight to the fifth weight is not limited here.

[0085] In another embodiment of the present application, if there are two groups of test code data, the initial quality scores corresponding to the two groups of test code data are respectively obtained through the first weight to the fifth weight; perform a mean operation on the initial quality scores corresponding to the two groups of test code data, and the result obtained is the code quality score of the model to be detected.

[0086] When there are multiple groups of test code data, the calculation formula for the code quality score is as follows:

[0087] ;

[0088] Among them, N is the number of groups of test code data, S final is the code quality score of the model to be detected; S i is the initial quality score corresponding to each group of test code data.

[0089] Specifically, if there are two groups of test code data, the first score of the first test code data is 8, the second score is 19, the third score is 7, the fourth score is 15, and the fifth score is 25; the first score of the second test code data is 10, the second score is 20, the third score is 6, the fourth score is 15, and the fifth score is also 25.

[0090] Therefore, the initial quality score S1 of the first test code data = 8 + 19 + 7 + 15 + 25 = 74; the initial quality score S2 of the second test code data = 10 + 20 + 6 + 15 + 25 = 76; furthermore, the quality score S of the model code to be detected final = (74 + 76) / 2 = 75.

[0091] The above code quality evaluation method realizes the code quality evaluation of the model to be detected through the test code data output by the model to be detected and the unilateral operation result after the unilateral operation of the test code data, and obtains the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score corresponding to the model to be detected. Furthermore, according to the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score, the code quality score of the model to be detected is determined. According to the above content, when the present application needs to evaluate the code quality of the model to be detected, it will pre-obtain the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score from five levels of modification ratio, code effectiveness, time efficiency, scenario diversity and code coverage respectively, so as to realize the comprehensive detection and evaluation of the test code data generated for the model to be detected, without requiring the operation and maintenance personnel to have a high level of professional knowledge, and ensuring the quality evaluation efficiency of the test code data output by the model to be detected.

[0092] In one embodiment, as Figure 3 shown, when it is necessary to evaluate the code quality of the model to be detected according to the test code data and obtain the modification ratio score corresponding to the model to be detected, the following contents may be included:

[0093] S301, extract the abnormal code data included in the test code data.

[0094] In one embodiment of the present application, when it is necessary to extract the abnormal code data included in the test code data, the test code data may be compiled or executed once to determine whether there is a situation where the compilation fails or a syntax error is reported during the execution of the test code data; if so, it is determined that there is abnormal code data in the test code data; the test code data is parsed through a code syntax analysis library to obtain the abnormal code data included in the test code data.

[0095] Furthermore, if it is determined that there is no situation where the compilation fails or a syntax error is reported during the execution of the test code data, it is determined that the modification ratio score is full marks.

[0096] S302, determine the modification ratio score corresponding to the model to be detected according to the number of abnormal characters corresponding to the abnormal code data and the total number of characters of the test code data.

[0097] It should be noted that when it is necessary to determine the modification ratio score corresponding to the model to be detected, the ratio between the number of abnormal characters in the abnormal code data and the total number of characters in the test code data can be used as the modification ratio score corresponding to the model to be detected.

[0098] Furthermore, if there is a situation where the test code data fails to compile or reports a syntax error during execution, the pre-trained modification model can be used to identify and modify the abnormal code data in the test code data, thereby obtaining the modification ratio score corresponding to the model to be detected.

[0099] In an embodiment of the present application, the test code data is compiled or executed once to determine whether there is a situation where the test code data fails to compile or reports a syntax error during execution; if so, the error message and the test code data are read, and the error message and the test code data are concatenated into an error prompt word. By inputting the error prompt word into the modification model, the ratio between the number of abnormal characters in the abnormal code data output by the modification model and the total number of characters in the test code data is the modification ratio score corresponding to the model to be detected.

[0100] The above code quality evaluation method realizes the determination of the modification ratio score corresponding to the model to be detected according to the number of abnormal characters in the abnormal code data and the total number of characters in the test code data through the abnormal code data included in the test code data, provides a data basis for determining the code quality score of the model to be detected subsequently, and ensures the smooth progress of the subsequent process.

[0101] In one embodiment, as Figure 4 shown, when it is necessary to perform code quality evaluation on the model to be detected according to the unilateral operation result to obtain the effectiveness score corresponding to the model to be detected, the following content may be included:

[0102] S401, obtain the standard defect code segments included in the preset benchmark test code dataset, and the total code amount corresponding to the standard defect code segments.

[0103] It should be noted that since the data format of the data in the benchmark test code dataset includes: programming language, code under test, usage example code, standard defect code segment, defect mark, and data; therefore, the standard defect code segments included in the benchmark test code dataset are identified and extracted.

[0104] Furthermore, a quantitative analysis is performed on the standard defect code segments included in the benchmark test code dataset to obtain the total code amount corresponding to the standard defect code segments.

[0105] S402, determine the defect code text description included in the unilateral operation result.

[0106] In an embodiment of the present application, the test code data running information included in the unilateral operation result is read and parsed to obtain the defective code text description included in the unilateral operation result.

[0107] S403, select the similar sub-code whose first similarity with the standard defective code segment is greater than the first threshold from the defective sub-code text descriptions included in the defective code text description.

[0108] In an embodiment of the present application, by comparing the defective sub-code text descriptions included in the defective code text description with the standard defective code segment respectively for similarity, the first similarity between each defective sub-code text description and the standard defective code segment is obtained; furthermore, the similar sub-code whose first similarity with the standard defective code segment is greater than the first threshold is selected from the defective sub-code text descriptions included in the defective code text description.

[0109] Among them, the first threshold can be set or adjusted according to the actual situation, and the value range of the first threshold is not limited here.

[0110] S404, determine the effectiveness score corresponding to the model to be detected according to the code quantity of the similar sub-code and the total code quantity.

[0111] It should be noted that when it is necessary to determine the effectiveness score corresponding to the model to be detected, the ratio of the code quantity of the similar sub-code to the total code quantity can be used as the effectiveness score corresponding to the model to be detected.

[0112] In an embodiment of the present application, when it is necessary to determine the effectiveness score corresponding to the model to be detected, the following contents may be included: determining the defective code text description included in the unilateral operation result; obtaining the standard defective code segment included in the preset benchmark test code dataset and the total code quantity corresponding to the standard defective code segment; setting the variable i = 0; first, when the total code quantity is zero, determining that the effectiveness score corresponding to the model to be detected is full marks; when the total code quantity is not zero, judging whether the total code quantity is greater than i, and when the total code quantity is greater than i, comparing the defective sub-code text description included in the defective code text description with the standard defective code segment for similarity to obtain the first similarity between the defective sub-code text description and the standard defective code segment; if the first similarity is greater than the first threshold (for example, 0.6), then controlling the count of the counter to increase by one, and controlling the variable i to increase by one, if the first similarity is not greater than the first threshold, then only controlling the variable i to increase by one and returning to execute the step of judging whether the total code quantity is greater than i; when the total code quantity is not greater than i, taking the ratio between the count of the counter and the total code quantity as the effectiveness score corresponding to the model to be detected.

[0113] In another embodiment of the present application, when it is necessary to determine the validity score corresponding to the model to be detected, the following may also be included: determining the text description of the defect code included in the unilateral operation result, and obtaining the standard defect code segment included in the preset benchmark test code dataset; furthermore, inputting the defect code text description, the standard defect code segment, the unilateral operation result, and the test code data into the analysis large model, and controlling the analysis large model to output the validity score corresponding to the model to be detected.

[0114] The above code quality evaluation method realizes determining the validity score corresponding to the model to be detected according to the number of codes of similar sub-codes and the total amount of codes by determining the number of codes of similar sub-codes and the total amount of codes, provides a data basis for determining the code quality score of the model to be detected subsequently, and ensures the smooth progress of the subsequent process.

[0115] In one embodiment, as Figure 5 shown, when it is necessary to perform code quality evaluation on the model to be detected according to the unilateral operation result to obtain the time efficiency score corresponding to the model to be detected, the following may be included:

[0116] S501, determining the average unilateral operation time of the test code data according to the total execution time and the number of unilateral operations corresponding to the unilateral operation result.

[0117] It should be noted that if the total execution time and the number of unilateral operations are included in the unilateral operation result, the content recorded in the unilateral operation result can be read to obtain the total execution time and the number of unilateral operations; if the total execution time and the number of unilateral operations are not included in the unilateral operation result, the total execution time and the number of unilateral operations are recorded during the unilateral operation of the test code data to obtain the total execution time and the number of unilateral operations.

[0118] In one embodiment of the present application, when it is necessary to determine the average unilateral operation time of the test code data according to the total execution time and the number of unilateral operations corresponding to the unilateral operation result, the total execution time and the number of unilateral operations can be subjected to a ratio operation, and the obtained result is the average unilateral operation time of the test code data.

[0119] S502, determining the time efficiency score corresponding to the model to be detected according to the average unilateral operation time and the preset time threshold.

[0120] It should be noted that when it is necessary to determine the time efficiency score corresponding to the model to be detected, the consumption rate corresponding to the model to be detected can be determined through the average unilateral operation time and the preset time threshold, and then the difference between the preset standard value and the consumption rate is used as the time efficiency score corresponding to the model to be detected.

[0121] Among them, the value range of the standard value can be set or adjusted according to the actual situation. For example, the standard value can be 1.

[0122] In an embodiment of the present application, when it is necessary to determine the time efficiency score corresponding to the model to be detected, if the standard value is 1, it is judged whether the average one-sided running time exceeds the time threshold. If it exceeds, the time-consuming rate is determined to be 1. At this time, the time efficiency score is equal to 1 - the time-consuming rate; if it does not exceed, the time-consuming rate is determined to be the ratio between the average one-sided running time and the time threshold. Therefore, the time efficiency score is equal to 1 - the ratio between the average one-sided running time and the time threshold.

[0123] The above code quality evaluation method realizes determining the time efficiency score corresponding to the model to be detected according to the average one-sided running time and the preset time threshold by determining the average one-sided running time of the test code data, provides a data basis for determining the code quality score of the model to be detected subsequently, and ensures the smooth progress of the subsequent process.

[0124] In an embodiment, as Figure 6 shown, when it is necessary to perform code quality evaluation on the model to be detected according to the test code data to obtain the diversity score corresponding to the model to be detected, the following contents may be included:

[0125] S601, obtain the standard diversity scenario data included in the preset benchmark test code dataset and the total number of scenarios corresponding to the standard diversity scenario data.

[0126] In an embodiment of the present application, by extracting data from the benchmark test code dataset, the standard diversity scenario data included in the benchmark test code dataset is obtained. Furthermore, by identifying the total number of scenarios included in the standard diversity scenario data, the total number of scenarios corresponding to the standard diversity scenario data is obtained.

[0127] S602, determine the test scenario data corresponding to the test code data.

[0128] It should be noted that when it is necessary to determine the test scenario data corresponding to the test code data, the test code data can be analyzed by the AST (Abstract Syntax Tree) of the code to extract the code comments of the test case function corresponding to the test code data, and the code comments representing the test scenario information are used as the test scenario data.

[0129] S603, select the similar sub-scenario data whose second similarity to the standard diversity scenario data is greater than the second threshold from the sub-scenario data included in the test scenario data.

[0130] It should be noted that if the second similarity between the sub-scenario data and the standard diversity scenario data is greater than the second threshold, it means that the test code data corresponding to the sub-scenario data has covered the corresponding standard diversity scenario data. Therefore, to ensure that the diversity score corresponding to the model to be detected can be successfully read subsequently, it is necessary to select from the sub-scenario data included in the test scenario data the similar sub-scenario data whose second similarity with the standard diversity scenario data is greater than the second threshold.

[0131] In one embodiment, if the number of sub-scenario data included in the test scenario data is 1, the similarity operation is performed between the sub-scenario data and at least one standard diversity scenario data to obtain the second similarity between the test scenario data and at least one standard diversity scenario data. Furthermore, if there is a standard diversity scenario data whose second similarity with the sub-scenario data is greater than the second threshold, the sub-scenario data is determined to be the similar sub-scenario data.

[0132] In another embodiment, if the number of sub-scenario data included in the test scenario data is multiple (i.e., there are multiple code comments), the similarity operation is performed between each sub-scenario data and at least one standard diversity scenario data to obtain the second similarity between each sub-scenario data and at least one standard diversity scenario data. For each sub-scenario data, if there is a standard diversity scenario data whose second similarity with the sub-scenario data is greater than the second threshold, the sub-scenario data is determined to be the similar sub-scenario data. Furthermore, all similar scenarios are realized.

[0133] S604. Determine the diversity score corresponding to the model to be detected according to the similarity quantity of the similar sub-scenario data and the total quantity of scenarios.

[0134] It should be noted that when it is necessary to determine the diversity score corresponding to the model to be detected according to the similarity quantity of the similar sub-scenario data and the total quantity of scenarios, the ratio between the quantity of scenarios and the total quantity of scenarios can be used as the diversity score corresponding to the model to be detected.

[0135] In an embodiment of the present application, when it is necessary to determine the diversity score corresponding to the model to be detected, the AST code abstract syntax tree analysis can be performed on the test code data to extract the code comments of the test case function corresponding to the test code data, and use the code comments representing the test scenario information as the test scenario data; obtain the standard diversity scenario data included in the preset benchmark test code dataset, and the total number of scenarios corresponding to the standard diversity scenario data; initialize the parameter i = 0, and initialize the similarity number of the similar sub-scenario data = 0; determine whether the total number of scenarios is greater than i. If the total number of scenarios is greater than i, determine the second similarity between a sub-scenario data and the standard diversity scenario data. If the second similarity between the sub-scenario data and the standard diversity scenario data is greater than the second threshold, the similarity number is incremented by one, and i is incremented by one, and then return to execute the step of determining whether the total number of scenarios is greater than i. If the total number of scenarios is not greater than i, use the number of scenarios of the similar sub-scenario data and the total number of scenarios as the diversity score corresponding to the model to be detected.

[0136] The above code quality evaluation method realizes determining the diversity score corresponding to the model to be detected according to the similarity number and the total number of scenarios of the similar sub-scenario data by determining the similarity number and the total number of scenarios of the similar sub-scenario data, provides a data basis for determining the code quality score of the model to be detected subsequently, and ensures the smooth progress of the subsequent process.

[0137] In one embodiment, as Figure 7 shown, when it is necessary to perform code quality evaluation on the model to be detected according to the unilateral operation result to obtain the coverage score corresponding to the model to be detected, the following may be included:

[0138] S701, extract the coverage value from the coverage report corresponding to the unilateral operation result.

[0139] It should be noted that if the coverage report is included in the unilateral operation result, directly obtain the coverage report included in the unilateral operation result; if the coverage report is not included in the unilateral operation result and it is necessary to extract the coverage value from the coverage report corresponding to the unilateral operation result, the coverage report corresponding to the unilateral operation result can be generated by a code coverage tool, and then, extract the coverage value from the coverage report.

[0140] S702, determine the coverage score corresponding to the model to be detected according to the coverage value.

[0141] In an embodiment of the present application, the coverage value can be used as the coverage score corresponding to the model to be detected.

[0142] The above code quality assessment method determines the coverage rate value, and based on the coverage rate value, determines the coverage rate score corresponding to the model to be detected, providing a data basis for subsequent determination of the code quality score of the model to be detected and ensuring the smooth progress of the subsequent process.

[0143] In one embodiment, as Figure 8 shown, when it is necessary to determine the code quality score of the model to be detected, the following content may be included:

[0144] S801, Obtain the test code data output by the model to be detected, and the unilateral operation result after the test code data is unidirectionally run;

[0145] S802, Extract the abnormal code data contained in the test code data; Determine the modification ratio score corresponding to the model to be detected according to the number of abnormal characters corresponding to the abnormal code data and the total number of characters of the test code data.

[0146] S803, Obtain the standard defect code segments contained in the preset benchmark test code dataset, and the total code volume corresponding to the standard defect code segments; Determine the defect code text description contained in the unilateral operation result; Select the similar sub-code with the first similarity greater than the first threshold between the defect sub-code text description contained in the defect code text description and the standard defect code segment; Determine the effectiveness score corresponding to the model to be detected according to the number of codes of the similar sub-code and the total code volume.

[0147] S804, Determine the average unilateral operation time of the test code data according to the total execution time and the number of unilateral operations corresponding to the unilateral operation result; Determine the time efficiency score corresponding to the model to be detected according to the average unilateral operation time and the preset time threshold.

[0148] S805, Obtain the standard diversity scenario data contained in the preset benchmark test code dataset, and the total number of scenarios corresponding to the standard diversity scenario data; Determine the test scenario data corresponding to the test code data; Select the similar sub-scenario data with the second similarity greater than the second threshold between the sub-scenario data contained in the test scenario data and the standard diversity scenario data; Determine the diversity score corresponding to the model to be detected according to the number of similarities of the similar sub-scenario data and the total number of scenarios.

[0149] S806, Extract the coverage rate value from the coverage report corresponding to the unilateral operation result; Determine the coverage rate score corresponding to the model to be detected according to the coverage rate value.

[0150] S807, Determine the code quality score of the model to be detected according to the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage rate score.

[0151] The above code quality evaluation method realizes the code quality evaluation of the model to be detected through the test code data output by the model to be detected and the unilateral operation results after the unilateral operation of the test code data, and obtains the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score corresponding to the model to be detected. Furthermore, according to the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score, the code quality score of the model to be detected is determined. According to the above content, when this application needs to evaluate the code quality of the model to be detected, it will pre-obtain the modification ratio score, effectiveness score, time efficiency score, diversity score and coverage score from five levels of modification ratio, code effectiveness, time efficiency, scenario diversity and code coverage respectively, so as to realize the comprehensive detection and evaluation of the test code data generated for the model to be detected, without requiring the operation and maintenance personnel to have a high level of professional knowledge, and ensuring the quality evaluation efficiency of the test code data output by the model to be detected.

[0152] It should be understood that although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0153] Based on the same inventive concept, the embodiments of this application also provide a code quality evaluation device for implementing the above-mentioned code quality evaluation method. The implementation solution provided by this device to solve the problem is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more embodiments of the code quality evaluation device provided below can refer to the limitations on the code quality evaluation method in the above text, and will not be repeated here.

[0154] In one embodiment, as Figure 9 shown, a code quality evaluation device is provided, including: an acquisition module 10, an evaluation module 20 and a determination module 30, where:

[0155] The acquisition module 10 is configured to acquire the test code data output by the model to be detected and the unilateral operation results after the unilateral operation of the test code data;

[0156] An evaluation module 20 is configured to evaluate the code quality of the model to be detected based on the test code data and / or the unilateral running result, and respectively obtain a modification ratio score, an effectiveness score, a time efficiency score, a diversity score, and a coverage score corresponding to the model to be detected;

[0157] A determination module 30 is configured to determine the code quality score of the model to be detected according to the modification ratio score, the effectiveness score, the time efficiency score, the diversity score, and the coverage score.

[0158] In one embodiment, the abnormal code data included in the test code data is extracted;

[0159] According to the number of abnormal characters corresponding to the abnormal code data and the total number of characters of the test code data, the modification ratio score corresponding to the model to be detected is determined.

[0160] In one embodiment, the standard defect code segments included in the preset benchmark test code dataset and the total amount of code corresponding to the standard defect code segments are obtained;

[0161] Determine the defect code text description included in the unilateral running result;

[0162] Select similar sub-codes with a first similarity greater than a first threshold between the defect sub-code text descriptions included in the defect code text description and the standard defect code segments;

[0163] According to the number of codes of the similar sub-codes and the total amount of code, the effectiveness score corresponding to the model to be detected is determined.

[0164] In one embodiment, according to the total execution time corresponding to the unilateral running result and the number of unilateral runs, the average unilateral running time of the test code data is determined;

[0165] According to the average unilateral running time and a preset time threshold, the time efficiency score corresponding to the model to be detected is determined.

[0166] In one embodiment, the standard diversity scenario data included in the preset benchmark test code dataset and the total amount of scenarios corresponding to the standard diversity scenario data are obtained;

[0167] Determine the test scenario data corresponding to the test code data;

[0168] Select similar sub-scenario data with a second similarity greater than a second threshold between the sub-scenario data included in the test scenario data and the standard diversity scenario data;

[0169] According to the number of similarities of the similar sub-scenario data and the total amount of scenarios, the diversity score corresponding to the model to be detected is determined.

[0170] In one embodiment, coverage values are extracted from the coverage report corresponding to the unilateral operation result;

[0171] According to the coverage values, a coverage score corresponding to the model to be detected is determined.

[0172] In one embodiment, a first weight corresponding to the modification ratio score, a second weight corresponding to the effectiveness score, a third weight corresponding to the time efficiency score, a fourth weight corresponding to the diversity score, and a fifth weight corresponding to the coverage score are determined;

[0173] The modification ratio score is weighted according to the first weight to obtain a first score;

[0174] The effectiveness score is weighted according to the second weight to obtain a second score;

[0175] The time efficiency score is weighted according to the third weight to obtain a third score;

[0176] The diversity score is weighted according to the fourth weight to obtain a fourth score;

[0177] The coverage score is weighted according to the fifth weight to obtain a fifth score;

[0178] The first score, the second score, the third score, the fourth score, and the fifth score are subjected to a sum operation to obtain the code quality score of the model to be detected.

[0179] The above-mentioned code quality evaluation device realizes the code quality evaluation of the model to be detected through the test code data output by the model to be detected and the unilateral operation result after the test code data is unilaterally run, and obtains the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage score corresponding to the model to be detected. Furthermore, according to the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage score, the code quality score of the model to be detected is determined. According to the above content, when the code quality evaluation of the model to be detected is required in this application, the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage score are respectively obtained from five levels of modification ratio, code effectiveness, time efficiency, scenario diversity, and code coverage, so as to realize the comprehensive detection and evaluation of the test code data generated for the model to be detected, without the need for operation and maintenance personnel to have a high level of professional knowledge, and ensure the quality evaluation efficiency of the test code data output by the model to be detected.

[0180] Each module in the above code quality evaluation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0181] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 10 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used for the processor to exchange information with external devices. The communication interface of the computer device is used for wired or wireless communication with external terminals, and the wireless method can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program, when executed by the processor, implements a code quality evaluation method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0182] Those skilled in the art can understand that Figure 10 the structure shown in

[0183] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0184] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0185] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A code quality evaluation method, characterized in that, The method includes: Obtaining the test code data output by the model to be detected, and the unilateral operation result after the unilateral operation of the test code data; Conducting code quality evaluation on the model to be detected according to the test code data and / or the unilateral operation result, and respectively obtaining the modification ratio score, effectiveness score, time efficiency score, diversity score, and coverage score corresponding to the model to be detected; Determining the code quality score of the model to be detected according to the modification ratio score, the effectiveness score, the time efficiency score, the diversity score, and the coverage score.

2. The method according to claim 1, wherein Conducting code quality evaluation on the model to be detected according to the test code data to obtain the modification ratio score corresponding to the model to be detected, including: Extracting the abnormal code data included in the test code data; Determining the modification ratio score corresponding to the model to be detected according to the number of abnormal characters corresponding to the abnormal code data and the total number of characters of the test code data.

3. The method according to claim 1, wherein Conducting code quality evaluation on the model to be detected according to the unilateral operation result to obtain the effectiveness score corresponding to the model to be detected, including: Obtaining the standard defect code segments included in the preset benchmark test code dataset, and the total amount of code corresponding to the standard defect code segments; Determining the defect code text description included in the unilateral operation result; Selecting similar sub-codes with a first similarity greater than a first threshold between the defect sub-code text descriptions included in the defect code text description and the standard defect code segments; Determining the effectiveness score corresponding to the model to be detected according to the number of codes of the similar sub-codes and the total amount of code.

4. The method according to claim 1, wherein Conducting code quality evaluation on the model to be detected according to the unilateral operation result to obtain the time efficiency score corresponding to the model to be detected, including: Determining the average unilateral operation time of the test code data according to the total execution time and the number of unilateral operations corresponding to the unilateral operation result; Determining the time efficiency score corresponding to the model to be detected according to the average unilateral operation time and a preset time threshold.

5. The method according to claim 1, characterized in that, Conducting code quality evaluation on the model to be detected according to the test code data to obtain the diversity score corresponding to the model to be detected, including: Obtaining the standard diversity scenario data included in the preset benchmark test code dataset, and the total amount of scenarios corresponding to the standard diversity scenario data; Determining the test scenario data corresponding to the test code data; Selecting similar sub-scenario data with a second similarity greater than a second threshold between the sub-scenario data included in the test scenario data and the standard diversity scenario data; Determining the diversity score corresponding to the model to be detected according to the number of similarities of the similar sub-scenario data and the total amount of scenarios.

6. The method according to claim 1, characterized in that, Conducting code quality evaluation on the model to be detected according to the unilateral operation result to obtain the coverage score corresponding to the model to be detected, including: Extracting the coverage value from the coverage report corresponding to the unilateral operation result; Determining the coverage score corresponding to the model to be detected according to the coverage value.

7. The method according to claim 1, wherein Determining the code quality score of the model to be detected according to the modification ratio score, the effectiveness score, the time efficiency score, the diversity score, and the coverage score includes: Determining a first weight corresponding to the modification ratio score, a second weight corresponding to the effectiveness score, a third weight corresponding to the time efficiency score, a fourth weight corresponding to the diversity score, and a fifth weight corresponding to the coverage score; Performing weighted processing on the modification ratio score according to the first weight to obtain a first score; Performing weighted processing on the effectiveness score according to the second weight to obtain a second score; Performing weighted processing on the time efficiency score according to the third weight to obtain a third score; Performing weighted processing on the diversity score according to the fourth weight to obtain a fourth score; Performing weighted processing on the coverage score according to the fifth weight to obtain a fifth score; Performing a sum operation on the first score, the second score, the third score, the fourth score, and the fifth score to obtain the code quality score of the model to be detected.

8. A code quality assessment device, characterized in that, The device includes: An acquisition module, configured to acquire test code data output by the model to be detected and the unilateral operation result after the test code data is unilaterally operated; An evaluation module, configured to perform code quality evaluation on the model to be detected according to the test code data and / or the unilateral operation result, and respectively obtain the modification ratio score, the effectiveness score, the time efficiency score, the diversity score, and the coverage score corresponding to the model to be detected; A determination module, configured to determine the code quality score of the model to be detected according to the modification ratio score, the effectiveness score, the time efficiency score, the diversity score, and the coverage score.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images