Software workload assessment method, electronic equipment, device, medium and program product

Abstract

The invention discloses a software workload evaluation method, electronic equipment, a device, a medium and a program product, and relates to the technical field of data processing, and the method comprises the steps: carrying out the pre-training of a to-be-trained model according to the basic knowledge of an international general evaluation standard, and obtaining an initial model; wherein the international general evaluation standard is COSMIC or NESMA, and the international general evaluation standard is COSMIC or Obtaining a training sample set, and performing iterative optimization on the initial model according to the training sample set to obtain a target model; inputting demand description information in a demand document of the to-be-evaluated software into the target model to obtain target function point information used for evaluating the workload of the to-be-evaluated software; and according to the demand description information and the target function point information in the demand document, generating a workload assessment table of the to-be-assessed software, and according to the workload assessment table, determining the workload of the to-be-assessed software. According to the method, the universality of the evaluation result can be improved while the evaluation efficiency and the evaluation accuracy of the software workload are improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a software workload assessment method, an electronic device, a device, a medium, and a program product. Background Art

[0002] Currently, when evaluating software workload, it usually requires professionals to spend a lot of time and effort interpreting and analyzing the software requirement documents to determine the function point information for evaluating the software workload, and then use the determined function point information to achieve the evaluation of the software workload.

[0003] However, this method not only has low evaluation efficiency, but is also easily interfered by the subjective factors of the evaluators, resulting in low evaluation accuracy. In addition, this evaluation method fails to combine with the international general evaluation standards, which will lead to poor universality of the evaluation results and limited application scope of the evaluation. Summary of the Invention

[0004] The main purpose of this application is to provide a software workload assessment method, an electronic device, a device, a medium, and a program product, aiming to improve the evaluation efficiency and accuracy of software workload while improving the universality of the evaluation results.

[0005] To achieve the above object, this application provides a software workload assessment method, and the method includes: Pre-train a model to be trained based on the basic knowledge of international general evaluation standards to obtain an initial model; wherein, the international general evaluation standard is COSMIC or NESMA; Obtain a training sample set, and iteratively optimize the initial model according to the training sample set to obtain a target model; wherein, the training sample set consists of multiple training samples, one training sample includes an input feature and a training label corresponding to the input feature, the input feature is constructed from a type of requirement description information, and the training label is constructed from a type of function point information; Input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain target function point information for evaluating the workload of the software to be evaluated; Generate a workload assessment form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, and determine the workload of the software to be evaluated according to the workload assessment form.

[0006] In an embodiment, the step of pre-training the model to be trained based on the basic knowledge of international general evaluation standards to obtain an initial model includes: Fine-tune the parameters of the model to be trained using the low-rank adaptation fine-tuning algorithm to obtain a fine-tuned model to be trained; Based on the basic knowledge of the internationally common evaluation criteria, pre-train the fine-tuned model to be trained to obtain the initial model.

[0007] In one embodiment, after the step of generating the workload evaluation form of the software to be evaluated according to the requirement description information in the requirement document and the target function point information, the method further includes: Obtain the target feedback information of the output result of the target model by artificial means; Generate a new training sample set based on the target feedback information; Incrementally train the target model according to the new training sample set to obtain a new target model.

[0008] In one embodiment, the step of obtaining the target feedback information of the output result of the target model by artificial means includes: Obtain the first feedback information of the artificial person for the workload evaluation form, and obtain the second feedback information of the evaluation experts of the internationally common evaluation criteria for each output result of the target model within a preset period; Summarize the first feedback information and the second feedback information to obtain the target feedback information; wherein, the target feedback information is used to generate the new training sample set for incremental training of the target model.

[0009] In one embodiment, the step of inputting the requirement description information in the requirement document of the software to be evaluated into the target model to obtain the target function point information for evaluating the workload of the software to be evaluated includes: Input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain the context information of the requirement description information in the requirement document through the target model; The target model determines the function point information based on the requirement description information in the requirement document and the context information to obtain the target function point information.

[0010] In one embodiment, each training sample in the training sample set further includes a thought chain corresponding to the input feature and the training label, and the thought chain is used to record the derivation process of deriving the training label corresponding to the input feature from the input feature; after the step of iteratively optimizing the initial model according to the training sample set to obtain the target model, the method further includes: Input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain the target function point information for evaluating the workload of the software to be evaluated, and output the target thought chain corresponding to the requirement description information in the requirement document and the target function point information.

[0011] In addition, to achieve the above object, the present application further provides a software workload assessment device, which includes: A model construction module, configured to pre-train a model to be trained based on the basic knowledge of international general assessment standards to obtain an initial model; wherein, the international general assessment standards are COSMIC or NESMA; obtain a training sample set, and based on the training sample set, iteratively optimize the initial model to obtain a target model; wherein, the training sample set consists of multiple training samples, one training sample includes an input feature and a training label corresponding to the input feature, the input feature is constructed from a type of requirement description information, and the training label is constructed from a type of function point information; A function point information determination module, configured to input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain target function point information for evaluating the workload of the software to be evaluated; A workload assessment module, configured to generate a workload assessment form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, and determine the workload of the software to be evaluated based on the workload assessment form.

[0012] In addition, to achieve the above object, the present application further provides an electronic device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the software workload assessment method as described above.

[0013] In addition, to achieve the above object, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the software workload assessment method as described above are implemented.

[0014] In addition, to achieve the above object, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the software workload assessment method as described above are implemented.

[0015] The present application provides a software workload assessment method. First, using the basic knowledge of international general assessment standards, the model to be trained is pre-trained so that the model masters the concepts and standards of the basic international general assessment standards, obtaining an initial model. Among them, the international general assessment standards are COSMIC or NESMA. Then, a training sample set is obtained to iteratively optimize the initial model using the training sample set to obtain a target model. Among them, the training sample set consists of multiple training samples. One training sample includes an input feature and a training label corresponding to the input feature. The input feature is constructed from a type of requirement description information, and the training label is constructed from a type of function point information. Since the initial model has mastered the concepts and standards of the basic international general assessment standards, when using the training sample set to iteratively train the initial model, its essence is to further strengthen the training of the model, thereby obtaining a target model with good performance for analyzing requirement description information to determine the corresponding function point information. After the construction of the target model is completed, by inputting the requirement description information in the requirement document of the software to be evaluated into the target model, the interpretation and analysis of the requirement document can be automatically completed by means of the target model, thereby obtaining the target function point information for evaluating the workload of the software to be evaluated. Then, by generating a workload assessment form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, the determination of the workload of the software to be evaluated can be completed using the workload assessment form.

[0016] Therefore, in summary, the technical solution of the present application performs two-stage training on the model to be trained by using the basic knowledge of international general assessment standards and the training sample set, thereby obtaining a target model with good performance for analyzing requirement description information to determine the corresponding function point information. Thus, the requirement document of the software to be evaluated can be automatically interpreted and analyzed by means of the target model, thereby realizing the determination of the workload of the software to be evaluated. Compared with the conventional method that requires manual interpretation and analysis, not only is the efficiency and accuracy higher, but the target model combines international general assessment standards during analysis, so the universality of the evaluation result of the software workload determined by the technical solution of the present application is also higher, and the application scope of the evaluation is wide. Therefore, the technical solution of the present application can improve the universality of the evaluation result while improving the evaluation efficiency and evaluation accuracy of the software workload. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application and, together with the specification, are used to explain the principles of the present application.

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a schematic flowchart of the software workload evaluation method provided in the first embodiment of the present application; Figure 2 It is a schematic flowchart of the extraction process of the requirement description information in the requirement document provided in the embodiment of the present application; Figure 3 It is a schematic flowchart of using the target model to complete the software workload evaluation provided in the embodiment of the present application; Figure 4 It is a schematic layout diagram of the front-end interface provided in the embodiment of the present application; Figure 5 It is a schematic module structure diagram of the software workload evaluation device provided in the embodiment of the present application; Figure 6 It is a schematic structural diagram of the hardware operating environment involved in the embodiment of the present application.

[0020] The implementation, functional features and advantages of the purpose of the present application will be further described in combination with the embodiments with reference to the accompanying drawings. Detailed implementation manners

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0022] To better understand the technical solutions of the present application, the following will be described in detail in combination with the accompanying drawings of the specification and specific implementation manners.

[0023] The execution subject of the software workload evaluation method of the present application can be an electronic device with data processing, network communication and program running functions, or a control system, control circuit, etc. that can implement the above functions, or a requirement management system. This embodiment does not make specific limitations on this.

[0024] The following takes the requirement management system as the execution subject as an example to illustrate the following embodiments.

[0025] Based on this, the present application proposes a software workload evaluation method for the first embodiment. Please refer to Figure 1 , the software workload evaluation method includes steps S10 to S40: Step S10, based on the basic knowledge of the international general evaluation standard, pre-train the model to be trained to obtain an initial model; among them, the international general evaluation standard is COSMIC or NESMA; It should be noted that the internationally common evaluation criteria can be COSMIC (Common Software Measurement International Consortium), NESMA (Netherlands Software Metrics Association), etc. This embodiment does not make specific limitations thereto. The basic knowledge of the internationally common evaluation criteria may include, but is not limited to, the core concepts for reference in evaluation (for example, COSMIC measures the function size by calculating the number of data movements in the software; NESMA quantifies software functions from the user's perspective based on function point analysis. These methods have clearly defined function point counting rules and scope boundaries, providing a unified framework for evaluation), knowledge related to function points, data movement types and processing logics, evaluation processes and steps, etc. This embodiment does not make specific limitations thereto.

[0026] In addition, it should be noted that in actual use, considering that models with parameters around 2B (billion) have simple logical capabilities and dialogue capabilities and are suitable for handling some classification tasks, such as the judgment of data movement types in COSMIC tasks. In contrast, models with parameters around 7B - 14B have stronger logical capabilities and can understand complex logics and output answers as required. Although these models may not be able to stably execute all complex tasks, through appropriate data fine-tuning, they can be successfully applied to production tasks, such as generating function processes according to requirement descriptions in COSMIC tasks. Thus, when selecting the model to be trained, models such as Llama3 - 8B (Llama model with 3 - 8 billion parameters) can be selected as the model to be trained.

[0027] The basic knowledge of the internationally common evaluation criteria can appear in the form of question-and-answer pair data. Specifically, refer to Table 1 below. Thus, the process of pre-training the model to be trained is essentially to use the question data in the question-and-answer pair data as the input of the model to be trained and the answer data in the question-and-answer pair data as the output of the model to be trained, and then the initial model can be trained.

[0028] Table 1:

[0029] It can be understood that by pre-training the model to be trained using the basic knowledge of international general evaluation criteria, the model can understand the definitions, uses, working principles, and application scenarios of international general evaluation criteria, so as to master the basic concepts and standards of international general evaluation criteria, thereby significantly enhancing its ability to handle tasks under international general evaluation criteria, facilitating the subsequent training to obtain a model with good performance.

[0030] In a feasible manner, to improve the performance of the finally constructed model in interpreting and analyzing software requirement documents, a low-rank adaptation fine-tuning algorithm can be used to fine-tune the parameters of the model to be trained. Specifically, step S10 may include steps S11 to S12: Step S11, use the low-rank adaptation fine-tuning algorithm to fine-tune the parameters of the model to be trained to obtain a fine-tuned model to be trained; Step S12, based on the basic knowledge of international general evaluation criteria, pre-train the fine-tuned model to be trained to obtain an initial model.

[0031] It should be noted that the low-rank adaptation (LoRA: Low-Rank Adaptation) fine-tuning algorithm is an efficient method for fine-tuning large language models, aiming to reduce the computational and storage requirements during the fine-tuning process while maintaining and improving the model's performance on specific tasks. Exemplarily, the hyperparameters of the LoRA fine-tuning algorithm can be configured as: learning rate 2e-5, batch size 16, number of training epochs 10, LoRA rank 8, weight decay 0.1.

[0032] Step S20, obtain a training sample set, and based on the training sample set, iteratively optimize the initial model to obtain a target model; where the training sample set consists of multiple training samples, one training sample includes an input feature and a training label corresponding to the input feature, the input feature is constructed from a type of requirement description information, and the training label is constructed from a type of function point information; It should be noted that the function point information at least includes one of first-level module, second-level module, third-level module, functional user, functional user requirements, triggering event, functional process, sub-process description, data movement type, data group, data attribute, and CFP (COSMIC Function Points). To facilitate the subsequent rapid generation of a workload evaluation form, the model can be trained to output function point information in a fixed format (such as JSON (JavaScript Object Notation) format). On this basis, taking the training label being constructed from the functional user and functional user requirements annotated in JSON format as an example, the training sample can be exemplarily referred to as Table 2 below: Table 2:

[0033] For another example, taking the construction of a functional process marked by a training label in JSON format as an example, the training samples can be exemplarily referred to Table 3 below: Table 3:

[0034] For yet another example, to achieve multi-task collaborative learning and improve the knowledge transfer ability of the model among different evaluation tasks, the training labels can be constructed from functional users, functional user requirements, and functional processes marked by JSON format, so that the training samples can be exemplarily referred to Table 4 below: Table 4:

[0035] Considering that when the model faces a large-demand document, it is necessary to adopt a step-by-step analysis method to determine the corresponding function point information. For example, first use the information in the requirement document to analyze the functional process, then use the functional process to analyze the functional subprocess, then use the functional subprocess to analyze the data movement type, and finally output the function point information including the functional process, functional subprocess, and data movement type. That is, decompose the model's task into multiple subtasks, so that multiple subtasks form a complete evaluation chain. For example, based on the actual requirements of the COSMIC evaluation standard, the task can be decomposed into five subtasks, and these five subtasks can form a complete evaluation chain: first determine "who" (functional user), then clarify "what to do" (functional process), then refine "how to do it" (subprocess), then judge "how the data flows" (data movement), and finally determine "when to do it" (trigger event).

[0036] That is: Subtask 1 (functional user judgment) identifies the functional user by analyzing the requirement description information and functional user rules and fills it into column G of the evaluation form. For example, for a requirement description like "The input person submits worker attendance and various expense information and confirms it", the functional user can be identified as "personnel administrator".

[0037] Subtask 2 (functional process extraction) extracts the complete functional process based on the requirement description information and the identified functional user and fills it into column I of the evaluation form. For example, the above example will be extracted as a functional process like "worker attendance expense entry".

[0038] Sub-task 3 (Sub-process description generation): For the identified functional processes, combined with the requirement description information, generate standardized sub-process step descriptions. For example, "Worker attendance expense entry" can be refined into specific steps such as "1. Enter attendance information; 2. Fill in expense details; 3. System verification; 4. Confirm and submit".

[0039] Sub-task 4 (Data movement type judgment): Based on the sub-process description and system operation type rules, identify the sub-process operation type (such as Entry (input) / Exit (output) / Read (read) / Write (write)) and fill it into column K of the evaluation form. An operation like "Enter attendance information" will be judged as "E", that is, the Entry type.

[0040] Sub-task 5 (Trigger event judgment): By analyzing the functional process description and requirement context, identify the trigger conditions of the functional process. For example, the trigger event of "Worker attendance expense entry" can be identified as "The personnel administrator initiates the attendance entry operation".

[0041] It can be understood that through the task decomposition strategy, dividing the complex evaluation process into multiple independent and interrelated sub-tasks can not only simplify the thinking link of each sub-task to significantly improve the model's understanding and processing ability, but also achieve seamless connection between sub-tasks through strictly defined input and output interfaces.

[0042] On this basis, to improve the model's ability to analyze each decomposed sub-task step by step, the training sample set can also include training samples whose input features are constructed from the description information of the functional process and whose training labels are constructed from the functional sub-processes marked in JSON format (which can be exemplarily referred to Table 5 below), and training samples whose input features are constructed from the description information of the functional sub-process and whose training labels are constructed from the data movement types marked in JSON format (which can be exemplarily referred to Table 6 below).

[0043] Table 5:

[0044] Table 6:

[0045] Among them, the data movement type of W type, that is, the data movement type of Write type.

[0046] For the step-by-step analysis of the model, to achieve multi-task collaborative learning to improve the model's knowledge transfer ability between different evaluation tasks, similarly, the training labels can be constructed from the functional sub-processes, data movement types, data groups, and data attributes marked in JSON format, so that the training samples can be exemplarily referred to Table 7 below: Table 7:

[0047] In a feasible implementation, to improve the accuracy of the model, when iteratively training the initial model using the training sample set, the training sample set can be first subjected to data cleaning processing to eliminate redundant data, duplicate data, incorrect data, etc. in the training sample set, so as to obtain a new training sample set; then the new training sample set is used to iteratively train the initial model to obtain the target model.

[0048] In addition, to improve the training efficiency of the model, during the model training process, the training methods of gradient accumulation and mixed precision can also be used, and an early stopping strategy is implemented to avoid model overfitting. Among them, when implementing the early stopping strategy, an indicator (such as accuracy rate, that is, the similarity rate of the result output by the model compared to the true result) can be selected as the target indicator to evaluate the performance of the model; then after each training cycle is completed, the validation sample set is used to determine the indicator value of the target indicator of the trained model; if the change range of the indicator value of the target indicator is less than the set amplitude threshold in multiple consecutive training cycles, the training is stopped. Among them, during the model training process, each time the indicator value of the target indicator is determined, the current indicator value of the target indicator determined in the current training cycle is compared with the historical indicator value of the target indicator determined in the previous training cycle. If the current indicator value is greater than the historical indicator value, the model trained in the current training cycle is stored; finally, after the model stops training, the stored model is used as the finally trained model.

[0049] Furthermore, a model performance monitoring panel can also be established in the requirements management system to real-time track the change trend of key indicators during the model training process, and a performance alarm threshold can be set as the basis for evaluating whether the performance of the model is good.

[0050] Additionally, during the model training process, contrastive learning can also be adopted to enhance the feature representation ability, thereby enhancing the model's recognition ability for similar requirement scenarios.

[0051] Step S30, input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain the target function point information for evaluating the workload of the software to be evaluated; It should be noted that the requirement document of the software to be evaluated is used to record the development requirements of the software to be evaluated. The requirement document usually includes information such as requirement descriptions, business rules, and business processes of the software project. The requirement description information in the requirement document of the software to be evaluated can be extracted by parsing the requirement document of the software to be evaluated through the document parsing module in the requirements management system. The extraction process can refer to Figure 2 , specifically: After receiving the requirement document of the software to be evaluated uploaded by the user, the document parsing module will extract the key information in the requirement document (such as system name, module name, requirement description, etc.) by parsing the document as requirement description information. Then, the document parsing module can call the target model according to the task type. Specifically, for the case where a detailed functional process task can be generated based on the requirement description information, the document parsing module will input the extracted requirement description information item by item into the target model to obtain the functional process in JSON format output by the model. For the case where a detailed functional subprocess description can be generated based on the functional process, the document parsing module will further input the received functional process item by item into the target model to obtain the functional subprocess description in JSON format output by the model; afterwards, by integrating the requirement document information and the target function point information containing various output results of the model, the document parsing module can generate a workload evaluation form for the software to be evaluated and return it to the user.

[0052] Step S40: Generate a workload evaluation form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, and determine the workload of the software to be evaluated based on the workload evaluation form.

[0053] It should be noted that in the process of generating a workload evaluation form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, it may include but is not limited to processes such as filling in tables and merging cells, and this embodiment does not make specific limitations on this.

[0054] This embodiment provides a software workload assessment method. First, using the basic knowledge of international general assessment standards, the model to be trained is pre-trained so that the model masters the concepts and standards of the basic international general assessment standards, obtaining an initial model. Among them, the international general assessment standards are COSMIC or NESMA. Then, a training sample set is obtained to iteratively optimize the initial model using the training sample set to obtain a target model. Among them, the training sample set consists of multiple training samples. One training sample includes an input feature and a training label corresponding to the input feature. The input feature is constructed from a type of requirement description information, and the training label is constructed from a type of function point information. Since the initial model has mastered the concepts and standards of the basic international general assessment standards, when using the training sample set to iteratively train the initial model, its essence is to further strengthen the training of the model, thereby obtaining a target model with good performance for analyzing requirement description information to determine the corresponding function point information. After the construction of the target model is completed, by inputting the requirement description information in the requirement document of the software to be evaluated into the target model, the interpretation and analysis of the requirement document can be automatically completed by means of the target model, thereby obtaining the target function point information for evaluating the workload of the software to be evaluated. Then, by generating a workload assessment form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, the determination of the workload of the software to be evaluated can be completed using the workload assessment form.

[0055] Therefore, in summary, this embodiment performs two-stage training on the model to be trained by using the basic knowledge of international general assessment standards and the training sample set, thereby obtaining a target model with good performance for analyzing requirement description information to determine the corresponding function point information. Thus, the requirement document of the software to be evaluated can be automatically interpreted and analyzed by means of the target model, thereby realizing the determination of the workload of the software to be evaluated. Compared with the conventional method that requires manual interpretation and analysis, not only is the efficiency and accuracy relatively high, but also the target model combines international general assessment standards in the analysis. Therefore, the universality of the evaluation result of the software workload determined in this embodiment is also relatively high, and the application scope of the evaluation is wide. Therefore, this embodiment can improve the universality of the evaluation result while improving the evaluation efficiency and accuracy of the software workload.

[0056] Based on the above first embodiment, a second embodiment of the software workload assessment method of this application is proposed. In the second embodiment, after step S40, the software workload assessment method may further include steps S50 to S70: Step S50, obtaining target feedback information on the output result of the target model by humans; It should be noted that the target feedback information may include, but is not limited to, the correction history of the output results of the target model by humans, the statistically common error types, the systematic biases of the analyzed model, etc. This embodiment does not make specific limitations on this.

[0057] In a feasible implementation manner, step S50 may include steps S51 to S52: Step S51, obtain the first feedback information of humans for the workload evaluation form, and obtain the second feedback information of evaluation experts of international general evaluation criteria for each output result of the target model within a preset period; It should be noted that the preset period may be a default period, or can be flexibly set by the user according to the actual situation. This embodiment does not make specific limitations on this.

[0058] Step S52, summarize the first feedback information and the second feedback information to obtain the target feedback information; wherein, the target feedback information is used to generate a new training sample set to perform incremental training on the target model.

[0059] This embodiment establishes a dual-track optimization mechanism. On the one hand, it will collect the first feedback information of humans for the workload evaluation form, and on the other hand, it will regularly invite evaluation experts of international general evaluation criteria to evaluate the output results of the target model to obtain the second feedback information; then, by summarizing the first feedback information and the second feedback information, the target feedback information is obtained. Thus, subsequently, both the first feedback information and the second feedback information will be used simultaneously to achieve incremental training of the target model to focus on strengthening error cases, thereby improving the model's ability to interpret and analyze the requirements documents of software.

[0060] This embodiment does not make specific limitations on the specific implementation manner of step S50. For example, in other feasible implementation manners, the first feedback information or the second feedback information may also be directly used as the target feedback information.

[0061] Step S60, generate a new training sample set based on the target feedback information; It should be noted that the new training sample set generated based on the target feedback information can be exemplarily referred to as Table 8 below: Table 8:

[0062] Step S70, perform incremental training on the target model according to the new training sample set to obtain a new target model.

[0063] In this embodiment, by means of the target feedback information of the output result of the target model by artificial means, a new training sample set is generated, and the target model is incrementally trained by using the new training sample set, so that the model can focus on strengthening the wrong cases, thereby improving the model's ability to interpret and analyze the requirement documents of the software.

[0064] Based on the above first embodiment and / or second embodiment, a third embodiment of the software workload assessment method of the present application is proposed. In the third embodiment, step S30 may include steps S31 to S32: Step S31, input the requirement description information in the requirement document of the software to be evaluated into the target model, so as to obtain the context information of the requirement description information in the requirement document through the target model; Step S32, the target model determines the function point information based on the requirement description information and the context information in the requirement document to obtain the target function point information.

[0065] It should be noted that the context information of the requirement description information refers to the background environment, constraints, influencing factors, etc. that are directly or indirectly related to the requirement, and they together constitute the boundary and applicable scenario of the requirement description.

[0066] In this embodiment, after the requirement description information in the requirement document of the software to be evaluated is input into the target model, the target model will first obtain the context information of the requirement description information in the requirement document, and then use the context information and the requirement description information in the requirement document at the same time to interpret and analyze the requirement document of the software, so that the target model can make full use of the relevant knowledge of the international general evaluation standard to determine the function point information, thereby accurately analyzing the target function point information and further improving the evaluation accuracy of the software workload.

[0067] The specific implementation manner of step S30 in this embodiment is not specifically limited. For example, in other feasible implementation manners, the target model may also determine the function point information only based on the requirement description information in the requirement document, so as to obtain the target function point information.

[0068] Based on the above first embodiment, second embodiment and / or third embodiment, a fourth embodiment of the software workload assessment method of the present application is proposed. In the fourth embodiment, each training sample in the training sample set further includes a thought chain corresponding to the input feature and the training label, and the thought chain is used to record the derivation process of deriving the training label corresponding to the input feature from the input feature; after step S20, the software workload assessment method may further include step S301: Step S301: Input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain the target function point information for evaluating the workload of the software to be evaluated, and output the target thought chain corresponding to the requirement description information and the target function point information in the requirement document.

[0069] It should be noted that the target thought chain is used to record the derivation process in which the target model infers the target function point information from the requirement description information in the input requirement document.

[0070] In this embodiment, by setting that each training sample in the training sample set further includes a thought chain corresponding to the input feature and the training label, when the target model constructed by using the training sample set interprets and analyzes the requirement document of the software, the target model will not only output the analysis result, that is, the target function point information, but also synchronously output the target thought chain used to record the derivation process in which the target model infers the target function point information from the requirement description information in the input requirement document, so as to improve the interpretability of the model output result by providing a detailed intermediate derivation process.

[0071] Exemplarily, to help understand how the user actually uses the target model to complete the software workload evaluation process, please refer to Figure 3 , specifically: The user logs in to the requirement management system to upload the requirement document of the software to be evaluated to the document parsing module through the front-end interface of the requirement management system; after receiving the requirement document, the document parsing module starts to parse the requirement document to extract the requirement description information in the requirement document and sends a request for an extraction task to the target model; after receiving the request for the extraction task, the target model will extract the target function point information for evaluating the workload of the software to be evaluated and return it to the document parsing module; after receiving the target function point information, the document parsing module will integrate the requirement description information and the target function point information and send a request for generating a workload evaluation form to the evaluation form generation module; after receiving the request, the evaluation form generation module will generate a workload evaluation form for the software to be evaluated according to the information integrated by the document parsing module and return it to the document parsing module; the document parsing module notifies the user that the workload evaluation form has been generated, and the user can download the workload evaluation form of the software to be evaluated through the front-end interface for evaluation; afterwards, the user can log in to the requirement management system again to upload the evaluated workload evaluation form to the corpus collection module; the corpus collection module receives the evaluated workload evaluation form, performs data screening and cleaning on it to obtain a high-quality data set, and uses the high-quality data set to iteratively optimize the target model to improve the model's ability, that is, to improve the evaluation accuracy and efficiency of the model.

[0072] Among them, the front-end interface of the requirement management system can be the interface layout as Figure 4 shown.

[0073] It should be noted that the above examples are only used to assist in understanding this application and do not constitute a limitation on the software workload assessment method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0074] An embodiment of this application also provides a software workload assessment device. Please refer to Figure 5 This device includes: A model construction module 10, configured to pre-train a model to be trained based on the basic knowledge of international general assessment standards to obtain an initial model. Among them, the international general assessment standards are COSMIC or NESMA. Obtain a training sample set, and based on the training sample set, iteratively optimize the initial model to obtain a target model. Among them, the training sample set is composed of multiple training samples. One training sample includes an input feature and a training label corresponding to the input feature. The input feature is constructed from a type of requirement description information, and the training label is constructed from a type of function point information. A function point information determination module 20, configured to input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain target function point information for evaluating the workload of the software to be evaluated. A workload assessment module 30, configured to generate a workload assessment form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, and determine the workload of the software to be evaluated based on the workload assessment form.

[0075] In one embodiment, the model construction module 10 is further configured to: Fine-tune the parameters of the model to be trained using a low-rank adaptation fine-tuning algorithm to obtain a fine-tuned model to be trained. Pre-train the fine-tuned model to be trained based on the basic knowledge of international general assessment standards to obtain an initial model.

[0076] In one embodiment, the software workload assessment device further includes a model optimization module, configured to: Obtain target feedback information of a human for the output result of the target model; Generate a new training sample set based on the target feedback information; Incrementally train the target model based on the new training sample set to obtain a new target model.

[0077] In one embodiment, the model optimization module is further configured to: Obtain first feedback information of a human for the workload assessment form, and obtain second feedback information of an assessment expert of international general assessment standards for each output result of the target model within a preset period; Summarize the first feedback information and the second feedback information to obtain target feedback information; wherein, the target feedback information is used to generate a new training sample set for incrementally training a target model.

[0078] In one embodiment, the function point information determination module 20 is further configured to: Input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain the context information of the requirement description information in the requirement document through the target model; Based on the requirement description information and the context information in the requirement document, the target model determines function point information to obtain target function point information.

[0079] In one embodiment, each training sample in the training sample set further includes a thought chain corresponding to the input feature and the training label together, and the thought chain is used to record the derivation process of deriving the training label corresponding to the input feature from the input feature; the software workload evaluation device further includes: Input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain target function point information for evaluating the workload of the software to be evaluated, and output a target thought chain corresponding to the requirement description information and the target function point information in the requirement document.

[0080] The software workload evaluation device provided by this application adopts the software workload evaluation method in the above embodiment, which can improve the evaluation efficiency and accuracy of the software workload while improving the universality of the evaluation result. Compared with the prior art, the beneficial effects of the software workload evaluation device provided by this application are the same as those of the software workload evaluation method provided by the above embodiment, and other technical features in the software workload evaluation device are the same as those disclosed in the method of the above embodiment, which will not be elaborated here.

[0081] An embodiment of this application further provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the software workload evaluation method in the above embodiment.

[0082] Next, refer to Figure 6 , which shows a schematic structural diagram of an electronic device suitable for implementing the embodiment of this application. Figure 6 The electronic device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiment of this application.

[0083] As Figure 6As shown, the electronic device may include a processing device 101 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in the read-only memory 102 or a program loaded from the storage device 103 into the random access memory 104. In the random access memory 104, various programs and data required for the operation of the electronic device are also stored. The processing device 101, the read-only memory 102, and the random access memory 104 are connected to each other through a bus 105. The input / output interface 106 is also connected to the bus 105. Generally, the following systems may be connected to the input / output interface 106: an input device 107 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 108 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 103 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 109. The communication device 109 may allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an electronic device having various systems, it should be understood that it is not required to implement or have all the shown systems, and alternatively, more or fewer systems may be implemented or had.

[0084] Specifically, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device, or installed from the storage device 103, or installed from the read-only memory 102. When the computer program is executed by the processing device 101, the above functions defined in the method of the embodiments of the present application are executed.

[0085] The electronic device provided by the embodiments of the present application, adopting the software workload evaluation method in the above embodiments, can improve the evaluation efficiency and accuracy of the software workload while improving the universality of the evaluation results. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiments of the present application are the same as those of the software workload evaluation method provided by the above embodiments, and other technical features in this electronic device are the same as those disclosed in the method of the above embodiments, and will not be elaborated here.

[0086] It should be understood that each part of the embodiments of the present application may be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in a suitable manner in any one or more embodiments or examples.

[0087] As described above, it is only the specific implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application shall be subject to the protection scope of the above-mentioned claims.

[0088] The embodiments of the present application further provide a computer-readable storage medium storing a computer program that can be run on a processor, and the computer program is used to execute the software workload assessment method in the above embodiments.

[0089] The computer-readable storage medium provided by the embodiments of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0090] The above computer-readable storage medium may be included in an electronic device; or it may exist separately without being assembled into the electronic device.

[0091] The above computer-readable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to: pre-train a model to be trained based on the basic knowledge of international general evaluation criteria to obtain an initial model; wherein, the international general evaluation criteria are COSMIC or NESMA; obtain a training sample set, and iteratively optimize the initial model based on the training sample set to obtain a target model; wherein, the training sample set consists of multiple training samples, one training sample includes an input feature and a training label corresponding to the input feature, the input feature is constructed from a type of requirement description information, and the training label is constructed from a type of function point information; input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain target function point information for evaluating the workload of the software to be evaluated; generate a workload evaluation form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, and determine the workload of the software to be evaluated based on the workload evaluation form.

[0092] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, execute as a stand-alone software package, execute partially on the user's computer and partially on a remote computer, or execute entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0094] The modules described in the embodiments of the present application can be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.

[0095] The computer-readable storage medium provided by the embodiments of the present application stores computer-readable program instructions for executing the above software workload assessment method, which can improve the assessment efficiency and accuracy of software workload while enhancing the universality of the assessment results. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiments of the present application are the same as those of the software workload assessment method provided by the above embodiments, and will not be elaborated here.

[0096] The embodiments of the present application also provide a computer program product, including a computer program, which when executed by a processor, implements the steps of the software workload assessment method as described above.

[0097] The computer program product provided by the embodiments of the present application can improve the assessment efficiency and accuracy of software workload while enhancing the universality of the assessment results. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiments of the present application are the same as those of the software workload assessment method provided by the above embodiments, and will not be elaborated here.

[0098] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural or equivalent process transformation made by using the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be similarly included in the patent scope of the present application.

Claims

1. A software workload assessment method, characterized in that The method includes: Pre-training the model to be trained based on the basic knowledge of international general evaluation criteria to obtain an initial model; wherein, the international general evaluation criteria are COSMIC or NESMA; Obtaining a training sample set, and iteratively optimizing the initial model according to the training sample set to obtain a target model; wherein, the training sample set consists of multiple training samples, one training sample includes an input feature and a training label corresponding to the input feature, the input feature is constructed from a kind of requirement description information, and the training label is constructed from a kind of function point information; Inputting the requirement description information in the requirement document of the software to be evaluated into the target model to obtain target function point information for evaluating the workload of the software to be evaluated; Generating a workload evaluation form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, and determining the workload of the software to be evaluated according to the workload evaluation form.

2. The software workload evaluation method according to claim 1, wherein, The step of pre-training the model to be trained based on the basic knowledge of international general evaluation criteria to obtain an initial model includes: Fine-tuning the parameters of the model to be trained using the low-rank adaptation fine-tuning algorithm to obtain a fine-tuned model to be trained; Pre-training the fine-tuned model to be trained based on the basic knowledge of the international general evaluation criteria to obtain the initial model.

3. The software workload assessment method according to claim 1, characterized in that, After the step of generating a workload evaluation form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, the method further includes: Obtaining target feedback information of an artificial person for the output result of the target model; Generating a new training sample set based on the target feedback information; Incrementally training the target model according to the new training sample set to obtain a new target model.

4. The software workload assessment method according to claim 3, wherein The step of obtaining target feedback information of an artificial person for the output result of the target model includes: Obtaining first feedback information of an artificial person for the workload evaluation form, and obtaining second feedback information of evaluation experts of the international general evaluation criteria for each output result of the target model within a preset period; Summarizing the first feedback information and the second feedback information to obtain the target feedback information; wherein, the target feedback information is used to generate the new training sample set for incrementally training the target model.

5. The software workload assessment method according to any one of claims 1 to 4, characterized in that, The step of inputting the requirement description information in the requirement document of the software to be evaluated into the target model to obtain target function point information for evaluating the workload of the software to be evaluated includes: Inputting the requirement description information in the requirement document of the software to be evaluated into the target model to obtain context information of the requirement description information in the requirement document through the target model; The target model determines function point information based on the requirement description information and the context information in the requirement document to obtain the target function point information.

6. The software workload assessment method according to any one of claims 1 to 4, characterized in that Each training sample in the training sample set further includes a thought chain corresponding to the input feature and the training label, where the thought chain is used to record the derivation process of deriving the training label corresponding to the input feature from the input feature; After the step of iteratively optimizing the initial model according to the training sample set to obtain a target model, the method further includes: Inputting the requirement description information in the requirement document of the software to be evaluated into the target model to obtain target function point information for evaluating the workload of the software to be evaluated, and outputting a target thought chain corresponding to the requirement description information in the requirement document and the target function point information.

7. A software workload assessment device, characterized in that, The device includes: A model construction module, configured to pre-train a model to be trained based on the basic knowledge of international general evaluation criteria to obtain an initial model; where the international general evaluation criteria are COSMIC or NESMA; obtaining a training sample set, and iteratively optimizing the initial model according to the training sample set to obtain a target model; where the training sample set is composed of multiple training samples, one training sample includes an input feature and a training label corresponding to the input feature, the input feature is constructed from a type of requirement description information, and the training label is constructed from a type of function point information; A function point information determination module, configured to input the requirement description information in the requirement document of the software to be evaluated into the target model to obtain target function point information for evaluating the workload of the software to be evaluated; A workload evaluation module, configured to generate a workload evaluation form for the software to be evaluated based on the requirement description information and the target function point information in the requirement document, and determine the workload of the software to be evaluated based on the workload evaluation form.

8. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the computer program is configured to implement the steps of the software workload evaluation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps of the software workload evaluation method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps of the software workload evaluation method according to any one of claims 1 to 6.

Images