A performance evaluation method, system and storage medium for software system

Abstract

The present application provides a performance evaluation method, system and storage medium for a software system, and relates to the field of data processing technology. The method includes creating an evaluation case based on a functional module, creating a corresponding evaluation program based on the evaluation case, executing the evaluation program to simulate user operation to obtain a first evaluation result, presetting a second execution result, modifying the functional module corresponding to the evaluation program that has different first and second execution results, repeatedly executing the evaluation program for the modified functional module to obtain a second evaluation result, training a first model to predict execution time, obtaining and optimizing functional modules that need to be improved based on the predicted execution time, obtaining related functional modules based on the correlation between the functional modules, collectively referring to the optimized functional modules and the related functional modules as first functional modules, executing the corresponding evaluation program on the first functional module to obtain a third evaluation result, thereby reducing the evaluation scope and improving the evaluation efficiency of the software system.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a performance evaluation method, system and storage medium for a software system. Background Art

[0002] With the rapid development of information technology, software systems are becoming more and more complex. Performance evaluation is a key step to ensure that software systems meet user needs and expected performance. Existing performance evaluation methods often rely on manual testing and empirical judgment, lack automation and accuracy, and cannot comprehensively and real-timely evaluate the performance of software systems.

[0003] The Chinese invention application with publication number CN104679655A provides a software system status evaluation method based on parameter correlation, comprising the following steps: step 1, determining the status parameters of the software system; step 2, determining the type of the status parameters, wherein the status parameters whose software system performance increases with the increase of the status parameter value are positively correlated parameters, and the status parameters whose software system performance decreases with the increase of the status parameter value are negatively correlated parameters; step 3, collecting the values ​​of all status parameters in the current software system and obtaining the measured value of each status parameter; step 4, scoring each status parameter according to the type, normal value and measured value of the status parameter; step 5, calculating the total score of all status parameters to determine the system status of the current software system. In addition, the Chinese invention application with publication number CN114780359A discloses a software performance quality evaluation method, device and storage medium, the method comprising: data collection: collecting historical data reflecting software performance quality to form a sample set, each sample including multiple performance quality indicator data; extracting several principal components based on the sample set by principal component analysis, and determining the principal component expression; constructing a principal component comprehensive model for evaluating software performance quality based on the extracted principal components; and evaluating the performance quality of the software to be evaluated according to the principal component comprehensive model. However, the above invention applications do not consider how to improve the evaluation efficiency, nor how to improve the performance of the software system based on the evaluation results. Based on this, a new technical solution is needed. Summary of the invention

[0004] In view of this, the embodiments of this specification provide a performance evaluation method, system and storage medium for a software system.

[0005] The embodiments of this specification provide the following technical solutions:

[0006] Step S1, creating corresponding evaluation cases based on the functional modules of the software system, creating corresponding evaluation programs based on the evaluation cases, executing the evaluation programs to simulate user operations, and recording a first evaluation result of executing each evaluation program, wherein the first evaluation result includes an execution time, an execution duration, a first execution result, and a resource utilization rate when executing the functional modules;

[0007] Step S2: preset a corresponding second execution result for each evaluation program, obtain the evaluation program whose first execution result and second execution result are different, obtain the corresponding functional module based on the evaluation program, modify the functional module, and repeatedly execute the evaluation program within a preset time period for the modified functional module to obtain the second evaluation result;

[0008] Step S3: training a first model based on the second evaluation result, wherein the first model can predict the execution time of each evaluation program under different resource utilization rates, obtaining the predicted execution time based on the first model, obtaining the functional modules that need to be improved based on the predicted execution time, and optimizing the functional modules that need to be improved;

[0009] Step S4, creating a correlation between each of the functional modules, obtaining and optimizing related functional modules related to the functional module based on the correlation, the optimized functional module and the related functional modules are collectively referred to as a first functional module, and within a preset time period, executing the corresponding evaluation program on the first functional module to obtain a third evaluation result.

[0010] Preferably, creating the correlation between the functional modules comprises the following steps:

[0011] Based on the software dependency analysis tool, the reference relationship between each of the functional modules is analyzed. For each of the functional modules, other functional modules that reference the functional module are taken as the related functional modules of the module. The functional modules whose corresponding first execution results in the second evaluation result of each functional module have abnormalities at the same time are counted. For the functional modules that have abnormalities at the same time, the first frequency of the first execution results having abnormalities at the same time is counted. The functional modules whose first frequency is greater than a preset first threshold are also taken as the related functional modules.

[0012] Preferably, obtaining the functional module to be improved based on the predicted execution time includes the following steps:

[0013] Step S31: acquiring a distribution range of the execution duration based on the second execution result, determining a plurality of duration intervals based on the distribution range, and allocating the execution result to the corresponding duration interval based on the execution duration of the second execution result;

[0014] Step S32: Count the first number of the second evaluation results contained in each of the duration intervals, obtain the duration interval where the maximum first number is located, refer to the duration interval as the first duration interval, and obtain all the execution durations of the first duration interval;

[0015] Step S33, obtain the first mode of all the execution times, obtain the first resource utilization rate corresponding to the first mode, obtain the first predicted execution time under the first resource utilization rate, compare whether the first mode is less than or equal to the first predicted execution time, if so, determine that the corresponding functional module does not need to be improved, determine whether the resource configuration of the software system needs to be optimized, if not, determine that the corresponding functional module needs to be improved.

[0016] Preferably, determining whether it is necessary to optimize the resource configuration of the software system comprises the following steps:

[0017] Step S331, determining whether the first duration interval is a maximum duration interval, where the maximum duration interval refers to the duration interval where the maximum execution duration is located. If yes, terminate this step;

[0018] Step S332, if no, obtain all the execution durations in the maximum duration interval, obtain the second mode of the execution durations, also obtain the second resource utilization rate corresponding to the execution durations, obtain the second predicted execution duration under the second resource utilization rate, calculate the first change rate of the first mode and the second mode, calculate the second change rate of the first predicted duration and the second predicted duration, and if the second change rate is less than the first change rate, determine that the resource configuration of the software system needs to be optimized.

[0019] Preferably, creating corresponding evaluation cases based on the functional modules of the software system includes the following steps:

[0020] Step S11, obtaining a requirement document, a program code and a modification record of the program code of the software system, associating each of the functional modules with the corresponding program code based on the requirement document, and determining evaluation angles based on the program code and the functional modules, wherein the number of the evaluation angles is greater than or equal to one;

[0021] Step S12, collecting relevant data for the evaluation angles, quantifying the evaluation angles based on the relevant data to determine corresponding first evaluation values, obtaining the corresponding first evaluation values ​​for each evaluation angle, and calculating the evaluation value of each functional module based on the first evaluation values, wherein the evaluation value is used to evaluate the risk level of each functional module;

[0022] Step S13: setting a plurality of different evaluation cases for the functional modules whose evaluation values ​​are greater than a preset evaluation threshold, wherein the evaluation cases include input data, execution steps and expected results.

[0023] Preferably, obtaining the first evaluation value corresponding to each evaluation angle comprises the following steps:

[0024] The relevant data refers to the first modification times of the program code within the past preset first time period, and the result value of dividing the first modification times by the total number of times the program code is submitted is used as the first evaluation value of the evaluation angle.

[0025] Preferably, calculating the evaluation value of each of the functional modules comprises the following steps:

[0026] Step S121, setting a corresponding weight value for each evaluation angle, multiplying the weight value corresponding to each evaluation angle of each functional module and the corresponding quantized value, and then adding them to obtain a total evaluation value;

[0027] Step S122, obtaining a first abnormal number of times the program code corresponding to the functional module is found to be abnormal during the review process, and dividing the first abnormal number by the total abnormal number corresponding to the functional module as a second evaluation value;

[0028] Step S123: Calculate an average value of the total evaluation value and the second evaluation value, and use the average value as the evaluation value of the functional module.

[0029] The present invention also provides a performance evaluation system for a software system, comprising the following modules:

[0030] A first evaluation unit is used to create a corresponding evaluation case based on the functional module of the software system, create a corresponding evaluation program based on the evaluation case, execute the evaluation program to simulate user operations, and record a first evaluation result of executing each evaluation program, wherein the first evaluation result includes an execution time, an execution duration, a first execution result, and a resource utilization rate when executing the functional module;

[0031] A second evaluation unit is used to preset a corresponding second execution result for each evaluation program, obtain the evaluation program whose first execution result and second execution result are different, obtain the corresponding functional module based on the evaluation program, modify the functional module, and repeatedly execute the evaluation program within a preset time period for the modified functional module to obtain the second evaluation result;

[0032] a function optimization unit, configured to train a first model based on the second evaluation result, wherein the first model can predict the execution time of each evaluation program under different resource utilization rates, obtain the predicted execution time based on the first model, obtain the function modules that need to be improved based on the predicted execution time, and optimize the function modules that need to be improved;

[0033] A third evaluation unit is used to create a correlation between each of the functional modules, obtain and optimize related functional modules related to the functional module based on the correlation, and refer to the optimized functional module and the related functional modules as a first functional module. Within a preset time period, the corresponding evaluation program is executed on the first functional module to obtain a third evaluation result.

[0034] The present invention also provides a storage medium, wherein the storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned methods for evaluating the performance of a software system.

[0035] Compared with the prior art, the at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0036] The present invention creates comprehensive evaluation cases based on multiple evaluation angles to ensure that the software system can meet the expected quality standards in all aspects. Functional modules that do not match the expected execution results are obtained through the first execution result, and these functional modules are modified. The second execution result is obtained by repeatedly executing the evaluation program within the budget time period. The first model is created based on the second execution result to predict the execution time. Based on the second execution result and the predicted execution time, it is determined which functional modules need to be improved.

[0037] Based on the judgment results, the functional modules that need to be improved are optimized accordingly, and the performance of the program code corresponding to the functional modules is improved in a targeted manner, thereby improving the performance of the overall software system; the functional modules related to the functional modules obtained and optimized based on the correlation are evaluated, which improves the targeted nature of the evaluation, reduces the scope of the evaluation, and improves the evaluation efficiency of the software system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a flowchart of the steps of the performance evaluation method for a software system in an embodiment of the present application;

[0040] Figure 2 It is a composition structure diagram of the performance evaluation system for the software system in the embodiment of the present application. DETAILED DESCRIPTION

[0041] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0042] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0043] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present application, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspect described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0044] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. The drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0045] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the examples can be practiced without these specific details.

[0046] For ease of understanding, the performance evaluation method for the software system in the embodiment of the present application is described below. Figure 1In the embodiment of the present application, a performance evaluation method for a software system is implemented by the following steps:

[0047] Step S1, create corresponding evaluation cases based on the functional modules of the software system, create corresponding evaluation programs based on the evaluation cases, execute the evaluation programs to simulate user operations, and record the first evaluation results of executing each evaluation program, the first evaluation results including the execution time, execution duration, the first execution result and the resource utilization rate when executing the functional modules.

[0048] Specifically, in order to perform performance evaluation on a software system, first, evaluation cases are created based on the functional modules of the software system. Traditional evaluation cases are mostly created based on the knowledge and experience of evaluators, which may lead to insufficient or incorrect evaluations. Therefore, it is necessary to improve the comprehensiveness and effectiveness of the evaluation. The specific process of creating evaluation cases will be explained in detail later. An evaluation program is created based on the created evaluation case, that is, corresponding evaluation codes are created based on the evaluation case. The evaluation program is executed to simulate various operations that a user may perform in the process of using the software system, and the first evaluation result of executing the evaluation program is recorded, wherein the execution time refers to the execution time point when the evaluation program is executed, the execution duration refers to the total duration used to execute the evaluation program, and the first execution result refers to the execution result output when the evaluation program is executed. Taking the software program as personnel management as an example, assuming that the evaluation program is to add personnel, the execution result is a successful addition or a failed addition. The resource utilization when executing the functional module refers to the CPU utilization, memory utilization, etc. when the evaluation program is executed.

[0049] Step S2: preset a corresponding second execution result for each evaluation program, obtain an evaluation program with different first execution results and second execution results, obtain a corresponding functional module based on the evaluation program, modify the functional module, and for the modified functional module, repeat the evaluation program within a preset time period to obtain a second evaluation result.

[0050] Specifically, the second execution result is the execution result that each evaluation program should output when there is no error in the function of the functional module. However, if there is a problem with the functional module, an execution result different from the expected one may appear when the evaluation program is used to evaluate the functional module. Therefore, by comparing the first execution result and the second execution result, an evaluation program with an execution result different from the expected one is obtained. When obtaining the functional module corresponding to the evaluation program, the problematic functional modules can be quickly obtained through the above method, and these functional modules can be modified. For the modified functional modules, the evaluation program is repeatedly executed to evaluate the functional modules within a preset time period, such as within one day. Subsequently, the software performance of the software program in different time periods is judged by the second evaluation result obtained.

[0051] Step S3: train the first model based on the second evaluation result, the first model can predict the execution time of each evaluation program under different resource utilization rates, obtain the predicted execution time based on the first model, obtain the functional modules that need to be improved based on the predicted execution time, and optimize the functional modules that need to be improved.

[0052] Specifically, under normal circumstances, a software system may have different visits in different time periods within a day. A large number of visits may affect the performance of the software system. In order to scientifically optimize the software system, the first model is trained based on the second evaluation result obtained. The execution time of the evaluation program can be predicted based on the first model under different resource utilization rates. The prediction results can be used to determine whether it is necessary to increase resource allocation for the software system. Or, when the resource utilization rate is low and the execution time is long, it may indicate that the program code of the corresponding functional module needs to be optimized. The functional modules that need to be improved can be accurately obtained based on the predicted execution time. The specific method for determining which functional modules need to be improved will be explained in detail later. After that, the functional modules that need to be improved can be optimized based on the judgment results. Through the above method, the performance of the program code corresponding to the functional module can be improved in a targeted manner, thereby improving the performance of the overall software system.

[0053] Step S4, create a correlation between the functional modules, obtain related functional modules related to the optimized functional modules based on the correlation relationship, and refer to the optimized functional modules and related functional modules as the first functional module. Within a preset time period, execute the corresponding evaluation program on the first functional module to obtain the third evaluation result.

[0054] Specifically, the software system will be continuously iterated and optimized over time, and the functional modules of the software system may also change continuously, but each iteration may only optimize one functional module. If all functional modules are evaluated once for each iteration and optimization of the software system, the efficiency of the evaluation will be reduced, and the evaluation time will also increase, resulting in slower evaluation efficiency. In order to improve the evaluation efficiency and effectively identify the evaluation content, reduce the evaluation scope, and shorten the evaluation time during the evaluation, after optimizing the software system, by creating a correlation between the functional modules, the relevant functional modules related to the optimized functional modules are obtained, and the optimized functional modules and the relevant functional modules are collectively referred to as first functional modules. Then, within a preset time period, the corresponding evaluation program is executed on the first functional module to obtain a third evaluation result, wherein the evaluation program is newly created based on the optimized functional module. After obtaining the third evaluation result, the relevant staff can evaluate the software system based on the third evaluation result, and can also perform the next iteration update of the software system based on the third evaluation result. The above method evaluates the functional modules related to the optimized functional modules based on the correlation acquisition, improves the pertinence of the evaluation, reduces the evaluation scope, and improves the evaluation efficiency of the software system.

[0055] Furthermore, creating the correlation between the functional modules includes the following steps:

[0056] The reference relationship between the functional modules is analyzed based on the software dependency analysis tool. For each functional module, other functional modules that reference the functional module are regarded as its own related functional modules. The functional modules whose corresponding first execution results in the second evaluation result of each functional module have abnormalities at the same time are also counted. For the functional modules that have abnormalities at the same time, the first frequency of the first execution results having abnormalities at the same time is counted, and the functional modules whose first frequency is greater than a preset first threshold are also regarded as related functional modules.

[0057] Specifically, in order to quickly obtain and optimize functional modules related to functional modules, firstly, the reference relationship between each functional module is analyzed based on the software dependency analysis tool. In the process of program code editing, if a functional module has referenced other functional modules, reference information is usually added in front of the entry file. The software dependency analysis tool can obtain the referenced other functional modules based on the reference information, and the reference relationship between each functional module can be obtained based on the reference information, and the related functional modules of each functional module can also be obtained. In order to conduct a more in-depth evaluation of the software system, the functional modules with simultaneous evaluation exceptions are obtained based on the second execution result. The evaluation exception means that the first evaluation result is different from the expected second evaluation result. If two functional modules always have evaluation exceptions at the same time, it is considered that there is a correlation between the two functional modules. The first frequency refers to the number of times evaluation exceptions occur at the same time in two functional modules. Through the above method, the functional modules with simultaneous evaluation exceptions are also regarded as related functional modules to prevent insufficient evaluation in subsequent evaluations.

[0058] Furthermore, obtaining the functional modules that need to be improved based on the predicted execution time includes the following steps:

[0059] Step S31, obtaining a distribution range of the execution duration based on the second execution result, determining a plurality of duration intervals based on the distribution range, and allocating the execution result to corresponding duration intervals based on the execution duration of the second execution result;

[0060] Step S32: Count the first number of second evaluation results included in each duration interval, obtain the duration interval where the maximum first number is located, refer to the duration interval as the first duration interval, and obtain all execution durations of the first duration interval;

[0061] Step S33, obtain the first mode of all execution times, obtain the first resource utilization rate corresponding to the first mode, obtain the first predicted execution time under the first resource utilization rate, compare whether the first mode is less than or equal to the first predicted execution time, if so, determine that the corresponding functional module does not need to be improved, determine whether the resource configuration of the software system needs to be optimized, if not, determine that the corresponding functional module needs to be improved.

[0062] Specifically, in order to determine which functional modules need to be improved, since multiple evaluations are performed within a predetermined time period, multiple second execution results will be obtained, each of which includes an execution time. Based on the distribution range of all the obtained execution times, for example, the shortest execution time is 50ms and the longest execution time is 3000ms, the distribution range of the execution time is 50ms-3000ms. Assume that the distribution range of the execution time is divided into three time intervals, and the three time intervals are divided according to historical experience. For example, the execution time is mostly distributed in 5 00ms-1800ms, the execution time is divided into three time intervals of 50ms-500ms, 500ms-1800ms, and 1800ms-3000ms. Based on the execution time, each second evaluation result is respectively allocated to the corresponding time interval, and the first number of second evaluation results included in each time interval is obtained, and the first time interval corresponding to the maximum first number is obtained, and then all execution times corresponding to the first time interval are obtained, and the first mode of all execution times is obtained. The first mode is the one that appears the most times in the first time interval. The execution time of the first mode is taken as the execution time of the evaluation program under normal circumstances, and then the first resource utilization rate corresponding to the first mode is obtained based on the second evaluation result. The first model is used to predict the first predicted execution time corresponding to the first resource utilization rate. The first predicted execution time refers to the execution time that should be used to execute the evaluation program under the first resource utilization rate. When the first mode is less than or equal to the first predicted time, it means that the execution time of the corresponding functional module is relatively short, and it also means that the performance of the functional module is good. Therefore, it is judged that the corresponding functional module does not need to be improved. However, this only represents the situation when the execution time is in the first execution interval. There are also situations where the execution time is longer, such as an execution time of 1800ms-3000ms. This situation may be due to the performance degradation caused by too many users accessing at the same time. Therefore, further judgment is required, which will be explained in detail later. If not, it means that the execution time in the time interval represented by the first mode is relatively long, and the performance of the corresponding functional module may be poor. Therefore, the corresponding functional module needs to be optimized, such as optimizing the logic of the corresponding program code to improve the performance of the corresponding functional module.

[0063] Furthermore, determining whether it is necessary to optimize the resource configuration of the software system includes the following steps:

[0064] Step S331: determine whether the first duration interval is a maximum duration interval, where the maximum duration interval refers to a duration interval where the maximum execution duration is located. If yes, end this step.

[0065] Specifically, in the case of the first resource utilization rate corresponding to the first duration interval, the corresponding execution time is relatively short, and the maximum duration interval refers to the duration interval where the maximum execution time is located, which means that the longest execution time is relatively short, which indicates the performance of the software system, so there is no need to optimize the functional modules. Usually, the resource utilization rate corresponding to the longest execution time is also large. When the resource utilization rate is large, the execution time is also short, which means that the performance of the software system is good enough and the resource configuration is relatively appropriate. Therefore, there is no need to optimize the resource configuration of the software system.

[0066] Step S332, if no, obtain all execution durations in the maximum duration interval, obtain the second mode of the execution durations, obtain the second resource utilization rate corresponding to the execution durations, obtain the second predicted execution duration under the second resource utilization rate, calculate the first change rate between the first mode and the second mode, calculate the second change rate between the first predicted duration and the second predicted duration, and if the second change rate is less than the first change rate, determine that the resource configuration of the software system needs to be optimized.

[0067] Specifically, if no, it means that there is a longer execution time, and a longer execution time may correspond to a greater resource utilization rate. Therefore, all execution times corresponding to the maximum time interval are obtained and the second mode thereof is obtained, and then the corresponding second resource utilization rate is obtained, and the first model is used to predict the second predicted execution time under the second resource utilization rate, and the first change rate and the second change rate are calculated. The first change rate refers to the change rate of the actual execution time when the resource utilization rate increases, and the second change rate refers to the change rate that the execution time should have when the resource utilization rate increases. If the second change rate is less than the first change rate, it means that the execution time slows down too quickly when the resource utilization rate increases, which does not meet the expected effect. It may be caused by low resource allocation. Therefore, in this case, it is judged that the resource allocation of the software system needs to be optimized.

[0068] Furthermore, creating corresponding evaluation cases based on the functional modules of the software system includes the following steps:

[0069] Step S11, obtaining a requirement document, program code and modification record of the program code of the software system, correlating each functional module and the corresponding program code with each other based on the requirement document, and determining evaluation angles based on the program code and the functional module, wherein the number of evaluation angles is greater than or equal to one;

[0070] Step S12: collect relevant data for the evaluation angle, quantify the evaluation angle based on the relevant data to determine the corresponding first evaluation value, obtain the corresponding first evaluation value for each evaluation angle, and calculate the evaluation value of each functional module based on the first evaluation value, and the evaluation value is used to evaluate the risk level of each functional module;

[0071] Step S13: setting a plurality of different evaluation cases for the functional modules whose evaluation values ​​are greater than a preset evaluation threshold, wherein the evaluation cases include input data, execution steps and expected results.

[0072] Specifically, the requirement document is a document used to record the instructions for use of the software system and the functional details of each functional module. Based on the requirement document and the program code, the functional module and the corresponding program code can be associated. Then, different evaluation angles can be determined based on the functional details of each functional module and the corresponding program code. The evaluation angles include integrity angle, reliability angle and compatibility angle, etc. The functional integrity angle refers to whether the functional module has taken into account the implementation of all small functional points under the functional module, the reliability angle refers to whether the functional module is stable when running continuously for a long time, and the compatibility angle refers to whether the functional module can be executed under different models of equipment, etc. Relevant data is collected for each evaluation angle, and the evaluation angle is quantified based on the relevant data to obtain the corresponding first evaluation value, and the evaluation value of the functional module is calculated based on multiple first evaluation values. The larger the evaluation value, the higher the possibility of problems in the corresponding functional module. Therefore, multiple evaluation cases are set for the evaluation angles whose evaluation values ​​are greater than the preset evaluation threshold to ensure that each situation of the corresponding evaluation angle is considered. Through the above method, more comprehensive and effective evaluation cases can be generated, thereby ensuring that the software system can meet the expected quality standards in all aspects.

[0073] Furthermore, obtaining a corresponding first evaluation value for each evaluation angle includes the following steps:

[0074] The relevant data refers to the first modification times of the corresponding program code within the past preset first time period, and the result value of dividing the first modification times by the total number of program code submissions is used as the first evaluation value from the evaluation perspective.

[0075] Specifically, taking the functional integrity evaluation angle as an example, the quantification steps are explained. The first number of modifications refers to the number of modifications made to the program code due to missing functions within a preset time period in the past. The preset first time period in the past refers to the self-inspection stage of the developer, that is, the time from the completion of the development of the functional module to the submission for evaluation. This period of time is reserved for the developer's self-inspection time. If the number of modifications is large, it may indicate that the corresponding functional module has more vulnerabilities and the possibility of subsequent errors is relatively large. Therefore, the evaluation can be based on the number of modifications. Assuming the number of modifications is 2, the total number refers to the total number of code submissions during the development of the functional module. Assuming it is 8 times, the value obtained by 2 / 8 can be used as the first evaluation value.

[0076] Furthermore, the evaluation value of each functional module is calculated, including the following steps:

[0077] Step S121, setting a corresponding weight value for each evaluation angle, multiplying the weight value corresponding to each evaluation angle of each functional module and the corresponding quantized value, and then adding them to obtain a total evaluation value;

[0078] Step S122, obtaining the first abnormal number of times the program code corresponding to the functional module is found to be abnormal during the review process, and dividing the first abnormal number by the total abnormal number corresponding to the functional module as the second evaluation value;

[0079] Step S123: Calculate the average of the total evaluation value and the second evaluation value, and use the average as the evaluation value of the functional module.

[0080] Specifically, in different software systems, different evaluation angles may have different importance. Based on the actual situation of the software system, for example, if the software system is a specific software system for a small number of people and is not used by multiple people, the weight value of the compatibility angle can be set smaller. The total evaluation value is obtained through the above step S121. In the process of developing program software, certain specifications will be set for the program code, such as naming specifications and submission specifications. If the program code does not meet the specifications when submitting, an exception will be prompted and the program code cannot be submitted successfully. During the development process, the more times the corresponding program code is found to be abnormal, it means that the developer who developed the functional module may not be careful enough, so the possibility of problems with the corresponding functional module is higher. Therefore, the second evaluation value is calculated through the above step S122, and finally the average of the total evaluation value and the second evaluation value is taken as the evaluation value, so a more scientific evaluation value of the functional module can be obtained.

[0081] According to another aspect of the embodiment of the present invention, Figure 2 As shown, a performance evaluation system for a software system is also provided, including a first evaluation unit, a second evaluation unit, a function optimization unit and a third evaluation unit, which are used to implement a performance evaluation method for a software system as described above, and the specific functions of each module are as follows:

[0082] A first evaluation unit is used to create corresponding evaluation cases based on the functional modules of the software system, create corresponding evaluation programs based on the evaluation cases, execute the evaluation programs to simulate user operations, and record a first evaluation result of executing each evaluation program, the first evaluation result including execution time, execution duration, first execution result, and resource utilization when executing the functional modules;

[0083] A second evaluation unit is used to preset a corresponding second execution result for each evaluation program, obtain an evaluation program having different first execution results and second execution results, obtain a corresponding functional module based on the evaluation program, modify the functional module, and repeatedly execute the evaluation program within a preset time period for the modified functional module to obtain a second evaluation result;

[0084] A function optimization unit is used to train a first model based on the second evaluation result, the first model can predict the execution time of each evaluation program under different resource utilization rates, obtain the predicted execution time based on the first model, obtain the function module that needs to be improved based on the predicted execution time, and optimize the function module that needs to be improved;

[0085] The third evaluation unit is used to create a correlation between the functional modules, and obtain and optimize the related functional modules based on the correlation relationship. The optimized functional modules and related functional modules are collectively referred to as first functional modules. Within a preset time period, the corresponding evaluation program is executed on the first functional module to obtain a third evaluation result.

[0086] According to another aspect of an embodiment of the present invention, a storage medium is provided, wherein the storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned methods for evaluating the performance of a software system.

[0087] In summary, the present invention provides a performance evaluation method, system and storage medium for a software system, the method includes creating an evaluation case based on a functional module, creating a corresponding evaluation program based on the evaluation case, executing the evaluation program to simulate user operations to obtain a first evaluation result, presetting a second execution result, modifying the functional module corresponding to the evaluation program that has different first and second execution results, repeatedly executing the evaluation program for the modified functional module to obtain a second evaluation result, training a first model to predict the execution time, obtaining and optimizing the functional module that needs to be improved based on the predicted execution time, obtaining related functional modules based on the correlation between the functional modules, collectively referring to the optimized functional modules and related functional modules as first functional modules, and executing the corresponding evaluation program on the first functional module to obtain a third evaluation result. The present application can improve the performance evaluation efficiency of the software system.

[0088] In this specification, the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments described later, the description is relatively simple, and the relevant parts can be referred to the partial description of the previous embodiments.

[0089] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A performance evaluation method for a software system, characterized in that: The steps include: Step S1, creating corresponding evaluation cases based on the functional modules of the software system, creating corresponding evaluation programs based on the evaluation cases, executing the evaluation programs to simulate user operations, and recording a first evaluation result of executing each evaluation program, wherein the first evaluation result includes an execution time, an execution duration, a first execution result, and a resource utilization rate when executing the functional modules; Step S2: preset a corresponding second execution result for each evaluation program, obtain the evaluation program whose first execution result and second execution result are different, obtain the corresponding functional module based on the evaluation program, modify the functional module, and repeatedly execute the evaluation program within a preset time period for the modified functional module to obtain the second evaluation result; Step S3: training a first model based on the second evaluation result, wherein the first model can predict the execution time of each evaluation program under different resource utilization rates, obtaining the predicted execution time based on the first model, obtaining the functional modules that need to be improved based on the predicted execution time, and optimizing the functional modules that need to be improved; Step S4, creating a correlation between the functional modules, obtaining and optimizing related functional modules related to the functional modules based on the correlation, the optimized functional modules and the related functional modules are collectively referred to as first functional modules, and executing the corresponding evaluation program on the first functional modules within a preset time period to obtain a third evaluation result; Among them, creating corresponding evaluation cases based on the functional modules of the software system includes the following steps: Step S11, obtaining a requirement document, a program code and a modification record of the program code of the software system, associating each of the functional modules with the corresponding program code based on the requirement document, and determining evaluation angles based on the program code and the functional modules, wherein the number of the evaluation angles is greater than or equal to one; Step S12, collecting relevant data for the evaluation angles, quantifying the evaluation angles based on the relevant data to determine corresponding first evaluation values, obtaining the corresponding first evaluation values ​​for each evaluation angle, and calculating the evaluation value of each functional module based on the first evaluation values, wherein the evaluation value is used to evaluate the risk level of each functional module; Step S13, setting a plurality of different evaluation cases for the functional modules whose evaluation values ​​are greater than a preset evaluation threshold, wherein the evaluation cases include input data, execution steps and expected results; The creation of the correlation between the functional modules comprises the following steps: Based on the software dependency analysis tool, the reference relationship between each of the functional modules is analyzed. For each of the functional modules, other functional modules that reference the functional module are taken as the related functional modules of the module. The functional modules whose corresponding first execution results in the second evaluation result of each functional module have abnormalities at the same time are counted. For the functional modules that have abnormalities at the same time, the first frequency of the first execution results having abnormalities at the same time is counted. The functional modules whose first frequency is greater than a preset first threshold are also taken as the related functional modules.

2. The method according to claim 1, characterized in that Acquiring a functional module that needs to be improved based on the predicted execution time includes the following steps: Step S31: acquiring a distribution range of the execution duration based on the second execution result, determining a plurality of duration intervals based on the distribution range, and allocating the execution result to the corresponding duration interval based on the execution duration of the second execution result; Step S32: Count the first number of the second evaluation results contained in each of the duration intervals, obtain the duration interval where the maximum first number is located, refer to the duration interval as the first duration interval, and obtain all the execution durations of the first duration interval; Step S33, obtain the first mode of all the execution times, obtain the first resource utilization rate corresponding to the first mode, obtain the first predicted execution time under the first resource utilization rate, compare whether the first mode is less than or equal to the first predicted execution time, if so, determine that the corresponding functional module does not need to be improved, determine whether the resource configuration of the software system needs to be optimized, if not, determine that the corresponding functional module needs to be improved.

3. The method according to claim 2, characterized in that Determining whether it is necessary to optimize the resource configuration of the software system includes the following steps: Step S331, determining whether the first duration interval is a maximum duration interval, where the maximum duration interval refers to the duration interval where the maximum execution duration is located. If yes, terminate this step; Step S332, if no, obtain all the execution durations in the maximum duration interval, obtain the second mode of the execution durations, also obtain the second resource utilization rate corresponding to the execution durations, obtain the second predicted execution duration under the second resource utilization rate, calculate the first change rate of the first mode and the second mode, calculate the second change rate of the first predicted duration and the second predicted duration, and if the second change rate is less than the first change rate, determine that the resource configuration of the software system needs to be optimized.

4. The method according to claim 1, characterized in that Obtaining the first evaluation value corresponding to each evaluation angle includes the following steps: The relevant data refers to the first modification times of the program code within the past preset first time period, and the result value of dividing the first modification times by the total number of times the program code is submitted is used as the first evaluation value of the evaluation angle.

5. The method according to claim 1, characterized in that Calculating the evaluation value of each of the functional modules comprises the following steps: Step S121, setting a corresponding weight value for each evaluation angle, multiplying the weight value corresponding to each evaluation angle of each functional module and the corresponding quantized value, and then adding them to obtain a total evaluation value; Step S122, obtaining a first abnormal number of times the program code corresponding to the functional module is found to be abnormal during the review process, and dividing the first abnormal number by the total abnormal number corresponding to the functional module as a second evaluation value; Step S123: Calculate an average value of the total evaluation value and the second evaluation value, and use the average value as the evaluation value of the functional module.

6. A performance evaluation system for a software system, used to implement a performance evaluation method for a software system as claimed in any one of claims 1 to 5, characterized in that: Includes the following modules: A first evaluation unit is used to create a corresponding evaluation case based on the functional module of the software system, create a corresponding evaluation program based on the evaluation case, execute the evaluation program to simulate user operations, and record a first evaluation result of executing each evaluation program, wherein the first evaluation result includes an execution time, an execution duration, a first execution result, and a resource utilization rate when executing the functional module; A second evaluation unit is used to preset a corresponding second execution result for each evaluation program, obtain the evaluation program whose first execution result and second execution result are different, obtain the corresponding functional module based on the evaluation program, modify the functional module, and repeatedly execute the evaluation program within a preset time period for the modified functional module to obtain the second evaluation result; a function optimization unit, configured to train a first model based on the second evaluation result, wherein the first model can predict the execution time of each evaluation program under different resource utilization rates, obtain the predicted execution time based on the first model, obtain the function modules that need to be improved based on the predicted execution time, and optimize the function modules that need to be improved; A third evaluation unit is used to create a correlation between each of the functional modules, obtain and optimize related functional modules related to the functional module based on the correlation, and refer to the optimized functional module and the related functional modules as a first functional module. Within a preset time period, the corresponding evaluation program is executed on the first functional module to obtain a third evaluation result.

7. A storage medium, characterized in that: The storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute a performance evaluation method for a software system according to any one of claims 1 to 5.

Images