Method and device for determining evaluation index of software project

Abstract

The invention discloses a method and a device for determining an evaluation index of a software project. The method comprises the following steps: determining an investment change score, an expected investment completion rate score, an expected overall demand effect score and an earlier-stage and later-stage evaluation result score of a software project; determining a pre-evaluation index; determining an overall project progress condition score, an expected investment completion realization condition score, a carry-over condition score, an expected effect realization condition score and an overall user satisfaction score of the software project; according to the overall project progress condition score, the expected investment completion realization condition score, the carry-over condition score, the expected effect realization condition score and the overall user satisfaction score, determining a post-evaluation index; and determining a target evaluation index of the software project according to the pre-evaluation index and the post-evaluation index. The technical problem that the evaluation index of the software project of the telecommunication industry cannot be comprehensively and efficiently determined in the related technology is solved.

Description

Technical Field

[0001] The present application relates to the field of production information management, and in particular, to a method and device for determining evaluation indicators of a software project. Background Art

[0002] In the telecommunications industry, the evaluation system of application software projects is of great significance for evaluating investment effects, improving investment benefits, and optimizing resource allocation. Common evaluation methods include understanding the functions of the system through on-site investigation and observation, evaluating business indicators, user satisfaction and other performance through data collection and analysis, understanding their evaluation and feedback on information performance through expert methods, determining their own performance level in the industry through comparative analysis, and promoting the further development of information construction through comprehensive evaluation and forming an evaluation report.

[0003] For example, the invention patent with announcement number: CN201610225270 discloses a method for evaluating the comprehensive performance of power informatization, which includes: (1) determining the evaluation object and indicator system of the comprehensive performance of power informatization; (2) collecting relevant data according to the evaluation object; (3) using the collected relevant data to calculate the evaluation value of the evaluation object, including evaluating the construction quality to obtain the construction quality evaluation value; evaluating the operation level to obtain the operation level evaluation value; evaluating the application effectiveness to obtain the application effectiveness evaluation value; evaluating the economic benefit to obtain the economic benefit evaluation value.

[0004] For example, the invention patent with publication number: CN106355340A provides a method and system for evaluating telecommunication products, wherein the method includes: obtaining operation information of telecommunication products; determining index information and product type of the telecommunication products according to the operation information; and evaluating the operation of the telecommunication products according to the index information and the product type. The method and system for evaluating telecommunication products provided by the embodiments of the present invention can perform targeted evaluation of telecommunication products according to the types of telecommunication products, thereby improving the pertinence and accuracy of the evaluation.

[0005] The above technology has at least the following technical problems:

[0006] 1. Single evaluation object: Most existing software evaluation methods focus on the consideration of a single project, focusing only on certain aspects of the project itself, such as functional implementation, local performance, etc., while ignoring key dimensions such as the unified planning of the overall project and benefit evaluation. For example, when evaluating multiple software projects in the telecommunications industry, the synergy between the projects, the rationality of resource allocation, and the degree of contribution to the overall strategic goals of the enterprise were not considered from the perspective of the overall project group. This single project evaluation method cannot fully grasp the overall situation of the project and is difficult to meet the needs of enterprises for more comprehensive and in-depth evaluation of projects.

[0007] 2. Insufficient evaluation dimensions: The current evaluation method mainly focuses on the evaluation feedback of the construction results and benefits in the later stage of the project, and lacks consideration of the pre-evaluation results such as the expected results in the early stage of project construction. This leads to the lack of effective connection and feedback mechanism between project decision-making and implementation, acceptance and other links. For example, in the project decision-making stage, due to the lack of previous evaluation data as a basis, unreasonable project plans may be formulated; during the project implementation process, it is impossible to adjust the strategy in time according to the comparison between the previous expectations and the actual progress; during the project acceptance, it is also difficult to comprehensively measure whether the project has achieved the expected goals, thus affecting the overall management efficiency and quality of the project.

[0008] 3. Lack of evaluation methods for the telecommunications industry: Current evaluation methods are mainly focused on the power and other industries, and there are relatively few evaluation methods for the telecommunications industry. Moreover, when evaluating telecommunications industry software projects, the existing evaluation methods have a relatively single evaluation dimension, and often fail to fully consider the characteristics of telecommunications industry software projects, such as the real-time nature of telecommunications services, high concurrent processing capabilities, and huge and diverse user scale. For example, when evaluating telecommunications billing software, only evaluating from traditional functional and performance indicators cannot accurately reflect the software's ability to cope with real-time billing of massive users, complex package billing rules processing, and collaborative work with other telecommunications business systems, and cannot comprehensively evaluate the overall situation of telecommunications industry software projects in a targeted manner.

[0009] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0010] The present application provides a method and device for determining evaluation indicators of a software project, so as to at least solve the technical problem that the related art cannot comprehensively and efficiently determine the evaluation indicators of software projects in the telecommunications industry.

[0011] According to one aspect of the present application, a method for determining evaluation indicators of a software project is provided, including: determining an investment change score, an expected investment completion rate score, an expected overall demand effectiveness score, and a pre- and post-evaluation result score of the software project; determining pre-evaluation indicators based on the investment change score, the expected investment completion rate score, the expected overall demand effectiveness score, and the pre- and post-evaluation result scores; determining the overall project progress score, the expected investment completion score, the carry-over score, the expected effectiveness achievement score, and the overall user satisfaction score of the software project; determining post-evaluation indicators based on the overall project progress score, the expected investment completion score, the carry-over score, the expected effectiveness achievement score, and the overall user satisfaction score; and determining target evaluation indicators for the software project based on the pre- and post-evaluation indicators.

[0012] Optionally, the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and preliminary post-evaluation result score of the software project are determined, including: determining the investment change score based on the overall investment change rate and the current year's main business income growth rate, wherein the overall investment change rate is the difference between the current year's overall planned investment amount and the previous year's overall planned investment amount; determining the expected investment completion rate score based on the expected investment completion rate for the current year, wherein the expected investment completion rate for the current year is the ratio of the expected annual account amount to the annual budget total; determining the expected overall demand effectiveness score based on the actual number of expected overall demand effectiveness indicators that are reported and do not deviate from the plan and the number of expected overall demand effectiveness indicators proposed in the current plan; determining the post-evaluation score of the single project dimension for the first two years of this year, and taking the weighted sum of the post-evaluation scores of the single project dimension for the first two years of this year and the post-evaluation scores of the overall project dimension for the first two years of this year to obtain the preliminary post-evaluation result score.

[0013] Optionally, determine the post-evaluation scores of the single project dimensions for the first two years of this year, including: determining the scores of the construction quality sub-indicator management standardization, the construction quality sub-indicator business support, the construction quality sub-indicator capability openness, and the construction quality sub-indicator technical advancement; weighted summing the scores of the construction quality sub-indicator management standardization, the construction quality sub-indicator business support, the construction quality sub-indicator capability openness, and the construction quality sub-indicator technical advancement to obtain the first summation result; determining the scores of the application effect sub-indicator user satisfaction and the application effect sub-indicator user satisfaction score. The score of the effect sub-indicator user activity is weightedly summed up with the score of the application effect sub-indicator user satisfaction and the score of the application effect sub-indicator user activity to obtain the second summation result; determine the score of the benefit sub-indicator input-output ratio and the score of the benefit sub-indicator demand effectiveness realization rate, and weightedly sum the score of the benefit sub-indicator input-output ratio and the score of the benefit sub-indicator demand effectiveness realization rate to obtain the third summation result; weighted sum the first summation result, the second summation result and the third summation result to obtain the single project dimension post-evaluation score for the first two years of this year.

[0014] Optionally, determine the scores of the management standardization of the construction quality sub-indicator, the business support score of the construction quality sub-indicator, the capacity opening score of the construction quality sub-indicator, and the technical advancement score of the construction quality sub-indicator, including: according to whether the actual construction period in the frozen handover stage of the project demand is less than the standard construction period, whether the actual construction period in the feasibility study stage of the project is less than the standard construction period, whether the actual construction period in the design stage of the project is less than the standard construction period, whether the actual construction period in the construction stage of the project is less than the standard construction period, whether the actual construction period in the initial inspection stage of the project is less than the standard construction period, whether the actual construction period in the trial operation stage of the project is consistent with the standard construction period, and whether the actual construction period in the final inspection stage of the project is less than the standard construction period, determine the scores of the management standardization of the construction quality sub-indicator; according to the number of realized business requirements and The total number of business requirements raised in the project is used to determine the score of the business support degree of the construction quality sub-indicator; the score of the capability openness of the construction quality sub-indicator is determined based on the ratio of the number of open capabilities actually called in the post-evaluation stage to the number of capability openness reported in the project establishment stage and whether there are any new capabilities opened in this phase of the project; the score of the technical advancement of the construction quality sub-indicator is determined based on whether the project has a promotion effect inside and outside the enterprise after it is launched, whether the construction of this phase of the project generates invention patents, whether the construction of this phase of the project generates enterprise standards, whether the construction of this phase of the project generates industry standards, whether the construction of this phase of the project generates national standards, whether the construction of this phase of the project generates enterprise-level awards, whether the construction of this phase of the project generates provincial and ministerial awards, and whether the construction of this phase of the project wins national awards.

[0015] Optionally, determine the score of the application effect sub-indicator user satisfaction and the score of the application effect sub-indicator user activity, including: determine the score of the application effect sub-indicator user satisfaction according to the number of user satisfaction survey results with 0 stars, the number of user satisfaction survey results with 1 star, the number of user satisfaction survey results with 2 stars, the number of user satisfaction survey results with 3 stars, the number of user satisfaction survey results with 4 stars, and the number of user satisfaction survey results with 5 stars; determine the score of the application effect sub-indicator user activity according to the number of valid registered users who have logged into the system in the past six months and the number of registered users of the system; determine the score of the benefit sub-indicator input-output ratio and the score of the benefit sub-indicator demand effectiveness realization rate, including: determine the score of the benefit sub-indicator input-output ratio according to the total income amount at the evaluation time, the total investment amount over the years, and whether the project can provide data sources and measurement processes; determine the score of the benefit sub-indicator demand effectiveness realization rate according to the number of achieved effectiveness improvements in the post-evaluation stage and the total number of expected demand effectiveness proposed in the pre-evaluation stage.

[0016] Optionally, determine the overall project progress score, expected investment completion score, carryover score, expected results achievement score, and overall user satisfaction score of the software project, including: determining the overall project progress score based on the number of projects whose overall construction period does not exceed the standard construction period after the post-evaluation is completed and the number of projects participating in the post-evaluation; determining the expected investment completion score based on the actual annual account amount, the expected investment completion rate for the current year, and the total annual budget; determining the carryover score by dividing the annual overall carryover amount by the total investment amount at the annual point in time; determining the expected results achievement score based on the number of realized demand results in the overall post-evaluation stage and the total number of overall expected demand results proposed in the overall pre-evaluation stage; determining the overall user satisfaction score based on the number of projects participating in the post-evaluation project user satisfaction index score greater than or equal to the preset branch and the number of projects participating in the post-evaluation.

[0017] Optionally, after determining the target evaluation indicators of the software project, the method also includes: evaluating the software project according to the target evaluation indicators to obtain evaluation results, wherein the evaluation results include: excellent, good, passing or failing; for software projects with excellent evaluation results, increasing their investment amount in the planned total investment stage; for software projects with failing evaluation results, reducing their investment amount in the planned total investment stage.

[0018] According to another aspect of the present application, there is also provided a device for determining evaluation indicators of a software project, comprising: a first determination module, used to determine the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores of the software project; a second determination module, used to determine the pre-evaluation indicator based on the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores; a third determination module, used to determine the overall project progress score, expected investment completion score, carry-over score, expected effectiveness achievement score, and overall user satisfaction score of the software project; a fourth determination module, used to determine the post-evaluation indicator based on the overall project progress score, expected investment completion score, carry-over score, expected effectiveness achievement score, and overall user satisfaction score; and a fifth determination module, used to determine the target evaluation indicator of the software project based on the pre-evaluation indicator and the post-evaluation indicator.

[0019] According to another aspect of the present application, a non-volatile storage medium is provided, the storage medium including a stored program, wherein when the program is run, the device where the storage medium is located is controlled to execute the above method for determining the evaluation index of the software project.

[0020] According to another aspect of the present application, an electronic device is provided, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the above method for determining the evaluation index of a software project is executed when the program is run.

[0021] According to yet another aspect of the present application, a computer program is provided, wherein when the computer program is executed by a processor, the method for determining the evaluation index of the above software project is implemented.

[0022] According to another aspect of the present application, a computer program product is provided, which includes a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for determining the evaluation indicators of the above software project is implemented.

[0023] In the present application, the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores of the software project are determined; the pre-evaluation index is determined according to the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores; the overall project progress score, expected investment completion score, carry-over score, expected effectiveness achievement score, and overall user satisfaction score of the software project are determined; the post-evaluation index is determined according to the overall project progress score, expected investment completion score, carry-over score, expected effectiveness achievement score, and overall user satisfaction score; the target evaluation index of the software project is determined according to the pre-evaluation index and the post-evaluation index, thereby achieving the purpose of comprehensively and efficiently determining the evaluation index of the software project in the telecommunications industry, thereby achieving the technical effect of improving the management and control capabilities of the software project, and further solving the technical problem that the related technology cannot comprehensively and efficiently determine the evaluation index of the software project in the telecommunications industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 is a flow chart of a method for determining an evaluation index of a software project according to an embodiment of the present application;

[0026] Figure 2 This is a conceptual diagram of an overall dimension evaluation system for software projects in the telecommunications industry according to an embodiment of the present application;

[0027] Figure 3 This is a conceptual diagram of a single project dimension post-evaluation indicator system for a telecommunications industry software project according to an embodiment of the present application;

[0028] Figure 4 is a structural diagram of a device for determining evaluation indicators of a software project according to an embodiment of the present application;

[0029] Figure 5 It is a hardware structure block diagram of a computer terminal according to a method for determining evaluation indicators of a software project in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0033] Expected demand results: Expected demand results are the predictions of the performance improvement of investment and construction projects corresponding to business needs. It usually covers aspects such as construction quality and efficiency, social and economic benefits, business support capabilities, and system processing capabilities.

[0034] Capability indicators: Capability indicators belong to a category of evaluation indicators, which are divided into general capability indicators and characteristic capability indicators. General capability indicators are general capability standards that are generally applicable to investment and construction projects (application software) in different fields or scopes, and are usually used to describe business data such as project business volume, system performance, and user volume; characteristic capability indicators are capability indicators set based on the project's business characteristics, business needs, and technical relevance, and are usually used to express the project's positioning, core capabilities, and key features.

[0035] Construction quality: Construction quality refers to the comprehensive performance of the system or software platform in terms of performance, applicability, reliability, security, economy, and the degree of conformity with user needs and technical requirements throughout the entire life cycle of the investment and construction project.

[0036] Business Support System (BSS): mainly refers to a set of systems used by telecommunications operators to support business operations, including but not limited to customer relationship management, billing system, order management, product configuration and many other functions, used to ensure the normal operation of the operator's business and provide customers with high-quality service experience.

[0037] Operation Support System (OSS): A system mainly used to ensure efficient, stable and secure operation of communication networks. It focuses on the management and maintenance of network equipment, network resources and network performance to ensure the quality and reliability of communication services.

[0038] Management Support System (MSS): A system that mainly provides decision support for the management of an enterprise or organization. It includes office collaboration, human resources, finance and other systems, helping managers to formulate strategies, plan resources, monitor performance and many other management activities.

[0039] According to an embodiment of the present application, a method embodiment of a method for determining evaluation indicators of a software project is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] Figure 1 is a flowchart of a method for determining an evaluation index of a software project according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0041] Step S101, determine the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores of the software project.

[0042] Step S102, determining the pre-evaluation index according to the investment change score, the expected investment completion rate score, the expected overall demand effectiveness score, and the pre-evaluation and post-evaluation result scores.

[0043] Step S103, determining the overall project progress score, expected investment completion score, carry-over score, expected results achievement score, and overall user satisfaction score of the software project.

[0044] Step S104, determining the post-evaluation indicators based on the overall project progress score, the expected investment completion score, the carry-over score, the expected results achievement score, and the overall user satisfaction score.

[0045] Figure 2 is a conceptual diagram of an overall dimension evaluation system for a telecommunications industry software project according to an embodiment of the present application, such as Figure 2 As shown in the figure, the evaluation indicators include pre-evaluation indicators and post-evaluation indicators. The overall dimension evaluation system combines pre-evaluation and post-evaluation. When evaluating the overall rationality of the total investment in software project planning, the pre-evaluation results of individual projects need to be summarized and input, and some of the post-evaluation results will also be input simultaneously, so as to comprehensively consider the evaluation results of the two stages in the overall process, so as to more accurately analyze and judge the rationality of the total investment in planning, and provide a decision-making basis for the overall planning layout and investment allocation.

[0046] The pre-evaluation indicators include four dimensions: investment change, expected investment completion rate, expected overall demand effect, and pre-post-evaluation results. Among them, investment change refers to the investment change rate of the overall project investment amount of the current period compared with the overall planned investment amount of the previous period; the expected investment completion rate refers to the indicator of the expected annual overall investment completion target, which is the percentage of the expected annual account amount to the total annual budget; the expected overall demand effect refers to the overall investment effect and construction effect, and the pre-post-evaluation results refer to the comprehensive post-evaluation results of the single project dimension of the previous year and the post-evaluation results of the overall dimension of the previous year.

[0047] Post-evaluation indicators include overall project progress, expected investment completion, carry-over, expected results, and overall user satisfaction. Among them, the overall project progress refers to the overall average level of the project's construction progress in the seven engineering stages of demand freeze transfer, feasibility study, design, construction, initial inspection, trial operation, and final inspection; the expected investment completion refers to the overall investment completion rate of the annual budget, which is the annual budget accounting completion score; the carry-over refers to the overall investment completion progress, which is the overall carry-over score; the expected results refers to the realization of the total number of overall expected demand results proposed in the overall dimension pre-evaluation stage in the overall post-evaluation stage; overall user satisfaction refers to the overall user satisfaction with the use of the software system.

[0048] Step S105: determining the target evaluation index of the software project according to the pre-evaluation index and the post-evaluation index.

[0049] According to the pre-assessment index setting of the overall dimension of the telecommunications industry software project, the total score of the pre-assessment is 100 points. Among them: 90 points and above are "excellent" projects, 70-90 (inclusive) are "good" projects, 60-70 (inclusive) are "passing" projects, and 60 (inclusive) points or less are failing projects. The score calculation formula is as follows:

[0050] CORE PE =K IC ×IC+K SI ×SI+K OD ×OD+K RP ×RP

[0051] In the formula, CORE PE K is the overall evaluation score of the previous evaluation; IC is the weight value of investment change, IC is the score of investment change; K SI is the weight value of investment completion, SI is the score of indicator investment completion; K OD is the weight value of the overall expected demand effect, OD is the score of the overall expected demand effect; K RP is the weight value of the previous and subsequent evaluation results; RP is the score of the previous and subsequent evaluation results.

[0052] Preferably, after collecting the above-mentioned evaluation data, a machine learning algorithm is used to conduct an in-depth analysis of a large amount of historical project data collected (including data collected in the early and late stages of the project) to explore potential patterns and associations in the data. For example, cluster analysis is used to identify the characteristics and development trends of different types of projects; regression algorithms are used to predict future investment changes, user satisfaction and other indicators of the project. More accurate data analysis results and forecasting information can be obtained, such as predicting the trend of user satisfaction changes in the project in the next few stages, and discovering potential problems in advance. This provides a more forward-looking and accurate basis for project decision-making and enhances the company's ability to respond to project risks.

[0053] The data analysis and prediction capabilities based on artificial intelligence can detect possible problems in the project in advance, such as the downward trend in user satisfaction and the risk of investment overspending, so that enterprises have enough time to take countermeasures and reduce project risks. For example, if it is predicted in advance that the telecommunications package management software project may cause user loss due to market competition, the enterprise can optimize the package content and services in advance. In addition, according to the characteristics and data patterns of different projects, personalized decision-making suggestions are provided for each project. For example, for technologically innovative telecommunications software projects, the algorithm can focus on technological advancement indicators and provide targeted investment and development suggestions to improve the accuracy and adaptability of decision-making.

[0054] In some implementations, the first neural network model may be trained using historical target evaluation indicators and historical revenue amounts of the software project, wherein the first neural network model is used to predict future revenue amounts of the software project.

[0055] Since the task is to predict numerical values, you can consider using a regression-type neural network, such as a multilayer perceptron (MLP) or a long short-term memory network (LSTM). Determine the number of nodes in the input layer (equal to the number of evaluation indicators), design the number and size of hidden layers, and the structure of the output layer (usually one node that outputs the predicted return amount). Divide the dataset into a training set, a validation set, and a test set. Using the training set data, adjust the weights of the neural network through the backpropagation algorithm to minimize the error between the predicted return and the actual historical return. Evaluate the model performance on the validation set and adjust the hyperparameters (such as learning rate, batch size, regularization, etc.) to optimize the model. Test the generalization ability of the model on the test set to ensure that the model not only performs well on the training data, but can also accurately predict unseen data. Use appropriate evaluation indicators such as mean squared error (MSE), root mean squared error (RMSE), or coefficient of determination (R 2 ) to evaluate the model's predictive performance. Input the evaluation indicators of the current or planned software project into the trained first neural network model. The model will output the predicted future revenue amount to provide data support for project investment decisions. Based on the predicted revenue amount, the enterprise can evaluate the potential return of the project, optimize resource allocation, and adjust the project strategy to achieve maximum benefits.

[0056] Further, the first neural network model can be loaded into the memory, for example, the raw data of the first neural network model can be loaded from the non-volatile memory into the volatile memory so that the processor runs the first neural network model. The raw data of the first neural network model refers to unprocessed data, which usually includes parameters and structural data of the first neural network model. The structural data can be a calculation relationship based on the parameters, such as the forward propagation calculation relationship between the intermediate layers and the neurons. Specifically, the structural data can include the structure-related code of the first neural network model, such as the code for performing related calculations between the intermediate layers and the neurons. Preferably, an area for loading the first neural network model can be divided in the memory, which can include a structural data storage area and a parameter storage area. The structural data storage area is used to store structure-related codes, and the parameters referenced by it can point to the address of specific parameters in the parameter storage area through pointers. During the training process of the first neural network model, it may be necessary to update the parameters frequently, and the parameter values ​​in the parameter storage area can be updated.

[0057] According to the above steps, the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores of the software project are determined; the pre-evaluation index is determined according to the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores; the overall project progress score, expected investment completion score, carry-over score, expected effectiveness achievement score, and overall user satisfaction score of the software project are determined; the post-evaluation index is determined according to the overall project progress score, expected investment completion score, carry-over score, expected effectiveness achievement score, and overall user satisfaction score; the target evaluation index of the software project is determined according to the pre-evaluation index and the post-evaluation index, thereby achieving the purpose of comprehensively and efficiently determining the evaluation index of the software project in the telecommunications industry, thereby achieving the technical effect of improving the management and control capabilities of the software project.

[0058] The following Figure 1 The steps shown are exemplary and explanatory.

[0059] According to some optional embodiments of the present application, determining the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result score of a software project can be achieved by the following method:

[0060] The investment change score is determined based on the overall investment change rate and the current year's main business income growth rate, where the overall investment change rate is the difference between the current year's overall planned investment amount and the previous year's overall planned investment amount; the expected investment completion rate score is determined based on the expected investment completion rate for this year, where the expected investment completion rate for this year is the ratio of the expected annual account amount to the annual budget total; the expected overall demand effectiveness score is determined based on the actual number of expected overall demand effectiveness indicators that are reported and do not deviate from the plan and the number of expected overall demand effectiveness indicators proposed in the current plan; the post-evaluation scores of the single project dimension for the first two years of this year are determined, and the weighted sum of the post-evaluation scores of the single project dimension for the first two years of this year and the post-evaluation scores of the overall project dimension for the first two years of this year are taken to obtain the score of the previous post-evaluation results.

[0061] Specifically, the investment change score IC calculation formula is as follows:

[0062]

[0063] In the formula, C1 is the overall investment change rate, that is, the overall planned investment of the current year minus the overall planned investment of the previous year, divided by the overall planned investment of the previous year; C2 is the growth rate of the company's main business income in the current year.

[0064] The calculation formula of the expected investment completion rate score SI is as follows:

[0065] SI=C3×100

[0066] In the formula, C3 is the estimated investment completion rate for this year, that is, the estimated annual account amount divided by the total annual budget.

[0067] The expected overall demand effectiveness score OD calculation formula is as follows:

[0068]

[0069] In the formula, C4 is the number of expected overall demand effectiveness indicators that are actually reported and do not deviate from the plan; C5 is the number of expected overall demand effectiveness indicators proposed in the current plan.

[0070] The calculation formula of RP score of the preliminary and post-evaluation results is as follows:

[0071] RP=C SE ×K SE +C OE ×K OE

[0072] In the formula, C SE is the post-evaluation score of a single project dimension in the previous year, K SE C is the post-evaluation weight value of a single project dimension in the previous year; OE is the overall dimension post-evaluation score of the previous year, K OE It is the evaluation weight value of the overall dimension in the previous year.

[0073] Preferably, determining the post-evaluation scores of the single project dimensions of the first two years of this year can be achieved by the following method: determining the scores of the management standardization of the construction quality sub-indicator, the business support score of the construction quality sub-indicator, the capability openness score of the construction quality sub-indicator, and the technical advancement score of the construction quality sub-indicator; performing weighted summation on the scores of the management standardization of the construction quality sub-indicator, the business support score of the construction quality sub-indicator, the capability openness score of the construction quality sub-indicator, and the technical advancement score of the construction quality sub-indicator to obtain the first summation result; determining the score of the user satisfaction of the application effect sub-indicator As well as the score of the user activity of the application effect sub-indicator, the score of the user satisfaction of the application effect sub-indicator and the score of the user activity of the application effect sub-indicator are weighted summed to obtain the second summation result; determine the score of the input-output ratio of the benefit sub-indicator and the score of the required effectiveness realization rate of the benefit sub-indicator, and weighted sum the score of the input-output ratio of the benefit sub-indicator and the score of the required effectiveness realization rate of the benefit sub-indicator to obtain the third summation result; weighted sum the first summation result, the second summation result and the third summation result to obtain the post-evaluation score of the single project dimension for the first two years of this year.

[0074] It is understandable that the above single-project dimension post-evaluation scores can be calculated based on the single-project post-evaluation indicator system, such as Figure 3 As shown in the figure, the post-project evaluation index system includes three parts: construction quality, application effect, and economic and social benefits. Construction quality refers to the progress of the entire project process, the support of the project to the business, and the overall completion quality of the project; application effect refers to the actual application experience and effect of the project after the completion of the project construction as evaluated by the demand proposer; economic and social benefits refer to the comparison between the expected demand results reported in the pre-project evaluation stage and the actual completion of the project construction. The score calculation formula is as follows:

[0075]

[0076] In the formula, CORE SE It is the comprehensive evaluation score of a single project. CQ is the weight value of the construction quality indicator; CQ1 is the score of the management standardization of the construction quality sub-indicator, and k1 is the weight value of the management standardization; CQ2 is the score of the business support of the construction quality sub-indicator, and k2 is the weight value of the business support; CQ3 is the score of the capability opening of the construction quality sub-indicator, and k3 is the weight value of the capability opening; CQ4 is the score of the technological advancement of the construction quality sub-indicator, and k4 is the weight value of the technological advancement. AE is the weight value of the application effect indicator; AE1 is the score of the application effect sub-indicator user satisfaction, m1 is the weight value of user satisfaction; AE2 is the score of the application effect sub-indicator user activity, m2 is the weight value of user activity. K EB is the weight value of the economic and social benefits indicator; EB1 is the score of the input-output ratio of the economic and social benefits sub-indicator, and n1 is the weight value of the input-output ratio; EB2 is the score of the demand-effectiveness realization rate of the economic and social benefits sub-indicator, and n2 is the weight value of the demand-effectiveness realization rate.

[0077] It should be noted that the scores for the management standardization of construction quality sub-indicators, the business support scores for construction quality sub-indicators, the capacity openness scores for construction quality sub-indicators, and the technological advancement scores for construction quality sub-indicators can be determined in the following ways: determine the scores for the management standardization of construction quality sub-indicators based on whether the actual construction period in the frozen handover phase of the project is less than the standard construction period, whether the actual construction period in the feasibility study phase of the project is less than the standard construction period, whether the actual construction period in the design phase of the project is less than the standard construction period, whether the actual construction period in the construction phase of the project is less than the standard construction period, whether the actual construction period in the initial acceptance phase of the project is less than the standard construction period, whether the actual construction period in the trial operation phase of the project is consistent with the standard construction period, and whether the actual construction period in the final acceptance phase of the project is less than the standard construction period; determine the scores for the management standardization of construction quality sub-indicators based on the realized business needs. The score of business support for the sub-indicator of construction quality is determined based on the number of requests and the total number of business requirements raised in the project establishment. The score of capability openness for the sub-indicator of construction quality is determined based on the ratio of the number of open capabilities actually called in the post-evaluation stage to the number of capabilities reported in the project establishment stage and whether there are any new capabilities opened in this phase of the project. The score of technical advancement for the sub-indicator of construction quality is determined based on whether the project has a promotion effect inside and outside the enterprise after it is launched, whether the construction of this phase of the project generates invention patents, whether the construction of this phase of the project generates enterprise standards, whether the construction of this phase of the project generates industry standards, whether the construction of this phase of the project generates national standards, whether the construction of this phase of the project generates enterprise-level awards, whether the construction of this phase of the project generates provincial and ministerial awards, and whether the construction of this phase of the project wins national awards.

[0078] Furthermore, determining the score of the application effect sub-indicator user satisfaction and the score of the application effect sub-indicator user activity can be achieved by the following method: determining the score of the application effect sub-indicator user satisfaction according to the number of user satisfaction survey results with 0 stars, the number of user satisfaction survey results with 1 star, the number of user satisfaction survey results with 2 stars, the number of user satisfaction survey results with 3 stars, the number of user satisfaction survey results with 4 stars, and the number of user satisfaction survey results with 5 stars; determining the score of the application effect sub-indicator user activity according to the number of valid registered users who have logged into the system in the past six months and the number of registered users of the system.

[0079] Furthermore, the score of the benefit sub-indicator input-output ratio and the score of the benefit sub-indicator demand effectiveness realization rate can be determined by the following method: determine the score of the benefit sub-indicator input-output ratio based on the total income amount at the evaluation time, the total investment amount over the years, and whether the project can provide data sources and calculation processes; determine the score of the benefit sub-indicator demand effectiveness realization rate based on the number of achieved effectiveness improvements in the post-evaluation stage and the total number of expected demand effectiveness proposed in the pre-evaluation stage.

[0080] Specifically, the standardization mainly evaluates whether the construction progress of the seven engineering stages from demand freeze handover, feasibility study, design, construction, initial inspection, trial operation and final inspection exceeds the standard construction period. The standard construction period is a reasonable construction time range formulated for a single project / engineering, which can be used as a reference for the planned construction period and a standard for macro-management of the engineering progress. The calculation formula for the score CQ1 is as follows:

[0081]

[0082] Wherein, T1 is whether the actual duration of the project demand freezing and handover stage is less than the standard duration. If so, the value is 1, if not, it is 0; T2 is whether the actual duration of the project feasibility study stage is less than the standard duration. If so, the value is 1, if not, it is 0; T3 is whether the actual duration of the project design stage is less than the standard duration. If so, the value is 1, if not, it is 0; T4 is whether the actual duration of the project construction stage is less than the standard duration. If so, the value is 1, if not, it is 0; T5 is whether the actual duration of the project initial inspection stage is less than the standard duration. If so, the value is 1, if not, it is 0; T6 is whether the actual duration of the project trial operation stage is consistent with the standard duration. If so, the value is 1, if not, it is 0; T7 is whether the actual duration of the project final inspection stage is less than the standard duration. If so, the value is 1, if not, it is 0.

[0083] Specifically, the business support mainly evaluates the realization of the number of business requirements proposed by the business requirement proposer in the project establishment stage in the post-evaluation stage. The score CQ2 calculation formula is as follows:

[0084]

[0085] In the formula, T8 is the number of realized business requirements, with the person who proposed the business requirements as the statistical standard; T9 is the total number of business requirements proposed in the project.

[0086] Specifically, capability opening mainly evaluates the actual call of the open capabilities (API, service interface) of the project. The score CQ3 calculation formula is as follows:

[0087] CQ3=T AC ×K AC +T NC ×K NC

[0088] Where, T AC K is the ratio of the number of open capabilities actually used in the post-evaluation stage to the number of open capabilities reported in the project establishment stage. AC is the corresponding weight value; T NC Indicates whether there is new capacity opening in this phase of the project. If yes, the value is 1, if not, it is 0. K NC is the corresponding weight value.

[0089] Specifically, the technological advancement mainly evaluates the promotion effect of the project inside and outside the enterprise after it is launched, as well as whether it has obtained invention patents, enterprise standards, industry standards, national standards, enterprise awards, provincial and ministerial awards, and national awards. The calculation formula for the score CQ4 is as follows:

[0090]

[0091] Where, T 10 After the project is launched, whether it has a promotion effect inside and outside the enterprise. If yes, the value is 1; if not, the value is 0. K 10 is the corresponding weight value; T 11 Whether the current project construction generates invention patents, if yes, the value is 1, if not, the value is 0, K 11 is the corresponding weight value; T 12 Indicates whether the current project construction generates enterprise standards. If yes, the value is 1; if no, the value is 0. K 12 is the corresponding weight value; T 13 Indicates whether the current project construction generates industry standards. If yes, the value is 1; if no, the value is 0. K 13 is the corresponding weight value; T 14 Indicates whether the current project construction generates national standards. If yes, the value is 1; if no, the value is 0. K 14 is the corresponding weight value; T 15 Indicates whether the current project construction generates enterprise-level awards. If yes, the value is 1; if no, the value is 0. K 15 is the corresponding weight value; T 16 Whether the current project construction has generated provincial and ministerial awards, if yes, the value is 1, if not, the value is 0, K 16 is the corresponding weight value; T 17 Whether the current project construction has won a national award, if yes, the value is 1, if not, the value is 0, K 17 is the corresponding weight value.

[0092] Specifically, user satisfaction mainly evaluates the user's satisfaction with the use of the software system. The score AE1 is calculated as follows:

[0093]

[0094] Where A1 is the number of user satisfaction survey results with 0 stars; A2 is the number of user satisfaction survey results with 1 star; A3 is the number of user satisfaction survey results with 2 stars; A4 is the number of user satisfaction survey results with 3 stars; A5 is the number of user satisfaction survey results with 4 stars; A6 is the number of user satisfaction survey results with 5 stars.

[0095] Specifically, user activity mainly evaluates the frequency of users' use of the software system. The calculation formula of score AE2 is as follows:

[0096]

[0097] In the formula, A7 is the number of valid registered users who have logged into the system in the past six months; A8 is the number of valid users of the system, that is, the number of registered users, excluding deleted or invalid users.

[0098] Specifically, the input-output ratio is the ratio of the income of the project with income to the total investment amount of the project after a series of calculations of the cumulative income amount and the cumulative actual investment amount of the project, so as to judge the investment benefit of the project. The calculation formula of the score EB1 is as follows:

[0099]

[0100] Wherein, E1 is the total income amount at the time of evaluation; E2 is the total investment amount over the years; E3 is whether the project can provide data source and calculation process, if yes, the value is 1, if not, the value is 0.

[0101] Specifically, the demand effectiveness realization rate mainly evaluates the realization of the total number of expected demand effectiveness proposed in the pre-evaluation stage in the post-evaluation stage, and is used to judge the investment effectiveness and construction effect of the project, and to evaluate whether the project has achieved business goals and efficiency improvement as expected. The calculation formula of the score EB2 is as follows:

[0102]

[0103] In the formula, E4 is the number of achieved results in the post-evaluation stage; E5 is the total number of expected demand results proposed in the pre-evaluation stage.

[0104] Preferably, determining the overall project progress score, expected investment completion score, carry-over score, expected results achievement score, and overall user satisfaction score of the software project can be achieved by the following method: determining the overall project progress score based on the number of projects whose overall construction period does not exceed the standard construction period after the post-evaluation is completed and the number of projects participating in the post-evaluation; determining the expected investment completion score based on the actual annual account amount, the expected investment completion rate for the current year, and the total annual budget; determining the carry-over score by dividing the annual overall carry-over amount by the total investment amount at the annual point in time; determining the expected results achievement score based on the number of achieved demand results in the overall post-evaluation stage and the total number of overall expected demand results proposed in the overall pre-evaluation stage; determining the overall user satisfaction score based on the number of projects participating in the post-evaluation project user satisfaction index score greater than or equal to the preset branch and the number of projects participating in the post-evaluation.

[0105] It should be noted that this embodiment is based on the post-evaluation index setting of the overall dimension of the telecommunications industry software project, and the total score of the post-evaluation is 100 points. Among them: 90 points and above are "excellent" projects, 70-90 (inclusive) are "good" projects, 60-70 (inclusive) are "passing" projects, and 60 (inclusive) points or less are failing projects. The score calculation formula is as follows:

[0106] CORE OE =K PP ×PP+K AC ×AC+K SC ×SC+K ER ×ER+K US ×US

[0107] In the formula, CORE OE It is the overall evaluation score of the post-evaluation. PP is the weight value of the overall project progress, PP is the score of the overall project progress; K AC is the weight value of the expected investment completion, AC is the score of the expected investment completion; K SC is the weight value of the carry-forward situation, SC is the score of the carry-forward situation; K ER is the weight value of the expected results, ER is the score of the expected results; K US is the weight value of overall user satisfaction, and US is the score of overall user satisfaction.

[0108] Specifically, the calculation formula for the overall project progress score PP is as follows:

[0109]

[0110] Where Q1 is the number of projects whose overall construction period does not exceed the standard construction period after the post-evaluation is completed; Q2 is the number of projects participating in the post-evaluation.

[0111] Specifically, the calculation formula for the expected investment completion score AC is as follows:

[0112]

[0113] In the formula, Q3 is the actual annual account amount; C3 is the expected investment completion rate for this year; Q4 is the total annual budget.

[0114] Specifically, the calculation formula for the transfer score SC is as follows:

[0115]

[0116] In the formula, Q5 is the annual overall carry-over rate, which is the annual overall carry-over amount divided by the total investment amount at the annual point in time.

[0117] Specifically, the expected results achievement score ER calculation formula is as follows:

[0118]

[0119] In the formula, Q6 is the number of realized demand results in the overall post-evaluation stage; Q7 is the total number of overall expected demand results proposed in the overall pre-evaluation stage.

[0120] Specifically, the overall user satisfaction score US is calculated as follows:

[0121]

[0122] Where, Q8 is the number of projects with user satisfaction index score AE1≥4 after participating in the post-evaluation project; Q9 is the number of projects participating in the post-evaluation.

[0123] Preferably, after determining the target evaluation indicators of the software project, the following steps can also be performed: evaluate the software project according to the target evaluation indicators to obtain evaluation results, wherein the evaluation results include: excellent, good, passing or failing; for software projects with excellent evaluation results, increase their investment amount in the planned total investment stage; for software projects with failing evaluation results, reduce their investment amount in the planned total investment stage.

[0124] For example, first, a series of quantitative target evaluation indicators need to be determined, including but not limited to the investment change rate, expected investment completion rate, expected demand effect realization rate, project progress deviation, user satisfaction, etc., to ensure that these indicators can fully reflect the execution of the software project and the degree of achievement of the expected goals. At the project start-up or planning stage, the pre-evaluation index of the software project is calculated based on the set indicators and the collected data. The evaluation formula may combine the weights and specific values ​​of each indicator, such as using weighted average, linear regression or machine learning models for comprehensive evaluation. During the project execution and at the end, collect project implementation and effect data to post-evaluate the software project. This will include the actual cost, benefits, user feedback, technical performance, etc. of the project. Compare the pre-evaluation and post-evaluation results of the software project with the preset evaluation criteria, and classify the project as excellent, good, qualified or unqualified. For example, a project with an evaluation result higher than 90 points can be rated as excellent, while a project with a score lower than 60 points can be rated as unqualified. According to the evaluation results, adjust the investment amount of the planning total investment stage. For software projects rated as excellent, because of their good execution and high expected returns, you can consider increasing the investment amount to further accelerate the project progress or expand the project scale to obtain greater returns. For software projects rated as unsatisfactory, due to execution problems, cost overruns or insufficient expected returns, the investment amount should be reduced to avoid waste of resources, and at the same time evaluate whether the project needs to be adjusted or terminated. For good and qualified projects, the current investment level can be maintained or adjusted appropriately according to the specific situation. Feedback the results of the investment decision adjustment to the project management team as a guide for project execution and optimization. At the same time, re-evaluate the project regularly, adjust the investment amount according to the latest situation, and form a cycle of continuous improvement.

[0125] Through the above process, enterprises can dynamically adjust investment and optimize resource allocation according to the actual performance and expected goals of software projects, ensure that funds are invested in projects with excellent performance, and intervene in poorly performing projects in a timely manner to improve the efficiency of the entire investment portfolio. This method relies on accurate evaluation indicators and scientific evaluation standards, as well as effective data analysis and decision-making mechanisms, and is an important innovation in software project management in the telecommunications industry.

[0126] Through the above steps, it can be seen that this application has the following advantages. Comprehensive and accurate evaluation of projects: By determining a comprehensive evaluation index system and collecting multi-source data for calculation, the project situation can be objectively and truly reflected from multiple dimensions. It avoids the one-sidedness of single indicator or partial dimension evaluation, so that enterprises have a more comprehensive and in-depth understanding of the project, so as to make more scientific and reasonable decisions. For example, when evaluating a telecommunications network optimization software project, not only technical indicators are considered, but also factors such as user satisfaction are integrated to ensure that the project meets the requirements in terms of technology and market acceptance. Optimize investment decisions: Adjust the investment amount according to the evaluation results. Increasing investment in excellent projects can further improve the project benefits. For unqualified projects, timely stop losses or improve them, which improves the return on investment and resource utilization efficiency. For example, a telecommunications company has increased investment in a 5G new business project that has been evaluated as excellent, accelerated market promotion, and obtained greater benefits; timely adjusted strategies for problem projects to avoid waste of resources. Improve project management level: The advantages and disadvantages of the project found during the evaluation process provide a clear direction for project improvement. Enterprises can optimize processes and strengthen management in a targeted manner to improve the overall quality and success rate of projects. For example, if it is found that a project scores low in terms of management standardization of construction quality, the standardization of project management processes can be strengthened to improve project quality.

[0127] Figure 4 is a structural diagram of a device for determining evaluation indicators of a software project according to an embodiment of the present application, such as Figure 4 As shown, the device comprises:

[0128] The first determination module 41 is used to determine the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result score of the software project.

[0129] The second determination module 42 is used to determine the pre-evaluation index according to the investment change score, the expected investment completion rate score, the expected overall demand effectiveness score, and the pre- and post-evaluation result scores.

[0130] The third determination module 43 is used to determine the overall project progress score, the expected investment completion score, the carry-over score, the expected results achievement score, and the overall user satisfaction score of the software project.

[0131] The fourth determination module 44 is used to determine the post-evaluation index according to the overall project progress score, the expected investment completion score, the carry-over score, the expected effect achievement score, and the overall user satisfaction score.

[0132] The fifth determination module 45 is used to determine the target evaluation index of the software project according to the pre-evaluation index and the post-evaluation index.

[0133] Optionally, the device for determining the evaluation index of the software project is also used to train a first neural network model using historical target evaluation indicators and historical revenue amounts of the software project, wherein the first neural network model is used to predict future revenue amounts of the software project.

[0134] Since the task is to predict numerical values, you can consider using a regression-type neural network, such as a multilayer perceptron (MLP) or a long short-term memory network (LSTM). Determine the number of nodes in the input layer (equal to the number of evaluation indicators), design the number and size of hidden layers, and the structure of the output layer (usually one node that outputs the predicted return amount). Divide the dataset into a training set, a validation set, and a test set. Using the training set data, adjust the weights of the neural network through the backpropagation algorithm to minimize the error between the predicted return and the actual historical return. Evaluate the model performance on the validation set and adjust the hyperparameters (such as learning rate, batch size, regularization, etc.) to optimize the model. Test the generalization ability of the model on the test set to ensure that the model not only performs well on the training data, but can also accurately predict unseen data. Use appropriate evaluation indicators such as mean squared error (MSE), root mean squared error (RMSE), or coefficient of determination (R 2 ) to evaluate the model's predictive performance. Input the evaluation indicators of the current or planned software project into the trained first neural network model. The model will output the predicted future revenue amount to provide data support for project investment decisions. Based on the predicted revenue amount, the enterprise can evaluate the potential return of the project, optimize resource allocation, and adjust the project strategy to achieve maximum benefits.

[0135] Further, the first neural network model can be loaded into the memory, for example, the raw data of the first neural network model can be loaded from the non-volatile memory into the volatile memory so that the processor runs the first neural network model. The raw data of the first neural network model refers to unprocessed data, which usually includes parameters and structural data of the first neural network model. The structural data can be a calculation relationship based on the parameters, such as the forward propagation calculation relationship between the intermediate layers and the neurons. Specifically, the structural data can include the structure-related code of the first neural network model, such as the code for performing related calculations between the intermediate layers and the neurons. Preferably, an area for loading the first neural network model can be divided in the memory, which can include a structural data storage area and a parameter storage area. The structural data storage area is used to store structure-related codes, and the parameters referenced by it can point to the address of specific parameters in the parameter storage area through pointers. During the training process of the first neural network model, it may be necessary to update the parameters frequently, and the parameter values ​​in the parameter storage area can be updated.

[0136] Optionally, the first determination module 41 is also used to perform the following steps: determine the investment change score based on the overall investment change rate and the current year's main business income growth rate, wherein the overall investment change rate is the difference between the current year's overall planned investment amount and the previous year's overall planned investment amount; determine the expected investment completion rate score based on the expected investment completion rate for this year, wherein the expected investment completion rate for this year is the ratio of the expected annual account amount to the annual budget total; determine the expected overall demand effectiveness score based on the actual number of expected overall demand effectiveness indicators that are not deviated from the plan and the number of expected overall demand effectiveness indicators proposed in the current plan; determine the post-evaluation scores of the single project dimensions for the first two years of this year, and take the weighted sum of the post-evaluation scores of the single project dimensions for the first two years of this year and the post-evaluation scores of the overall project dimensions for the first two years of this year to obtain the scores of the previous post-evaluation results.

[0137] Optionally, the first determination module 41 is also used to perform the following steps: determine the score of the construction quality sub-indicator management standardization, the score of the construction quality sub-indicator business support, the score of the construction quality sub-indicator capability openness, and the score of the construction quality sub-indicator technical advancement; perform weighted summation on the score of the construction quality sub-indicator management standardization, the score of the construction quality sub-indicator business support, the score of the construction quality sub-indicator capability openness, and the score of the construction quality sub-indicator technical advancement to obtain a first summation result; determine the score of the application effect sub-indicator user satisfaction and the score of the application effect sub-indicator user satisfaction. The score of the indicator user activity is weightedly summed up with the score of the application effect sub-indicator user satisfaction and the score of the application effect sub-indicator user activity to obtain the second summation result; determine the score of the benefit sub-indicator input-output ratio and the score of the benefit sub-indicator demand effectiveness realization rate, and weightedly sum the score of the benefit sub-indicator input-output ratio and the score of the benefit sub-indicator demand effectiveness realization rate to obtain the third summation result; weighted sum the first summation result, the second summation result and the third summation result to obtain the single project dimension post-evaluation score for the first two years of this year.

[0138] Optionally, the first determination module 41 is also used to perform the following steps: determine the score of the management standardization of the construction quality sub-indicator according to whether the actual construction period of the project demand freezing and handover stage is less than the standard construction period, whether the actual construction period of the project feasibility study stage is less than the standard construction period, whether the actual construction period of the project design stage is less than the standard construction period, whether the actual construction period of the project construction stage is less than the standard construction period, whether the actual construction period of the project initial inspection stage is less than the standard construction period, whether the actual construction period of the project trial operation stage is consistent with the standard construction period, and whether the actual construction period of the project final inspection stage is less than the standard construction period; determine the business support of the construction quality sub-indicator according to the number of realized business requirements and the total number of business requirements proposed in the project establishment. The score of the degree; the score of the capacity opening of the construction quality sub-indicator is determined according to the ratio of the number of open capabilities actually called in the post-evaluation stage to the number of open capabilities reported in the project establishment stage and whether there are any new open capabilities in this phase of the project; the score of the technical advancement of the construction quality sub-indicator is determined according to whether the project has a promotion effect inside and outside the enterprise after it is launched, whether the construction of this phase of the project generates invention patents, whether the construction of this phase of the project generates enterprise standards, whether the construction of this phase of the project generates industry standards, whether the construction of this phase of the project generates national standards, whether the construction of this phase of the project generates enterprise-level awards, whether the construction of this phase of the project generates provincial and ministerial awards, and whether the construction of this phase of the project wins national awards.

[0139] Optionally, the first determination module 41 is also used to perform the following steps: determine the score of the application effect sub-indicator user satisfaction according to the number of user satisfaction survey results with 0 stars, the number of user satisfaction survey results with 1 star, the number of user satisfaction survey results with 2 stars, the number of user satisfaction survey results with 3 stars, the number of user satisfaction survey results with 4 stars, and the number of user satisfaction survey results with 5 stars; determine the score of the application effect sub-indicator user activity according to the number of valid registered users who have logged into the system in the past six months and the number of registered users of the system; determine the score of the benefit sub-indicator input-output ratio and the score of the benefit sub-indicator demand effectiveness realization rate, including: determining the score of the benefit sub-indicator input-output ratio according to the total income amount at the evaluation time, the total investment amount over the years, and whether the project can provide data sources and measurement processes; determine the score of the benefit sub-indicator demand effectiveness realization rate according to the number of achieved effectiveness improvements in the post-evaluation stage and the total number of expected demand effectiveness proposed in the pre-evaluation stage.

[0140] Optionally, the third determination module 43 is also used to perform the following steps: determine the overall project progress score based on the number of projects whose overall construction period does not exceed the standard construction period after the post-evaluation is completed and the number of projects participating in the post-evaluation; determine the expected investment completion score based on the actual annual account amount, the expected investment completion rate of the current year and the annual total budget; determine the carryover score by dividing the annual overall carryover amount by the total investment amount at the annual point in time; determine the expected results realization score based on the number of realized demand results in the overall post-evaluation stage and the total number of overall expected demand results proposed in the overall pre-evaluation stage; determine the overall user satisfaction score based on the number of projects participating in the post-evaluation project user satisfaction index score greater than or equal to the preset branch and the number of projects participating in the post-evaluation.

[0141] Optionally, the device for determining the evaluation index of the software project is also used to perform the following steps after determining the target evaluation index of the software project: evaluating the software project according to the target evaluation index to obtain an evaluation result, wherein the evaluation result includes: excellent, good, passing or failing; for software projects with excellent evaluation results, increasing their investment amount in the planned total investment stage; for software projects with failing evaluation results, reducing their investment amount in the planned total investment stage.

[0142] It should be noted that the above Figure 4 The modules in the embodiment may be program modules (e.g., a set of program instructions for implementing a specific function) or hardware modules. For the latter, they may be expressed in the following forms, but are not limited thereto: the expression form of the above modules is a processor, or the functions of the above modules are implemented by a processor.

[0143] It should be noted that Figure 4 The preferred implementation of the illustrated embodiment can be found in Figure 1 The relevant description of the illustrated embodiment will not be repeated here.

[0144] Figure 5 The hardware structure block diagram of a computer terminal for implementing a method for determining evaluation indicators of a software project is shown. Figure 5 As shown, the computer terminal 50 may include one or more (502a, 502b, ..., 502n are used to illustrate) processors 502 (the processor 502 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 504 for storing data, and a transmission module 506 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 5 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 5 More or fewer components as shown, or with Figure 5 Different configurations shown.

[0145] It should be noted that the one or more processors 502 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 50. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0146] The memory 504 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for determining the evaluation index of the software project in the embodiment of the present application. The processor 502 executes various functional applications and data processing by running the software programs and modules stored in the memory 504, that is, the method for determining the evaluation index of the software project described above is realized. The memory 504 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 504 may further include a memory remotely arranged relative to the processor 502, and these remote memories may be connected to the computer terminal 50 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0147] The transmission module 506 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 50. In one example, the transmission module 506 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 506 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0148] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 50 .

[0149] It should be noted that, in some optional embodiments, the above Figure 5 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 5 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0150] It should be noted that Figure 5 The computer terminal shown is used to execute Figure 1 The method for determining the evaluation indicators of the software project shown in the figure, therefore the relevant explanations in the execution method of the above command are also applicable to the electronic device and will not be repeated here.

[0151] The embodiment of the present application further provides a non-volatile storage medium, the non-volatile storage medium includes a stored program, wherein when the program is run, the device where the storage medium is located is controlled to execute the above method for determining the evaluation index of the software project.

[0152] A program for performing the following functions on a non-volatile storage medium: determining the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores of a software project; determining pre-evaluation indicators based on the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores; determining the overall project progress score, expected investment completion score, carry-over score, expected effectiveness achievement score, and overall user satisfaction score of the software project; determining post-evaluation indicators based on the overall project progress score, expected investment completion score, carry-over score, expected effectiveness achievement score, and overall user satisfaction score; determining target evaluation indicators for the software project based on the pre-evaluation indicators and post-evaluation indicators.

[0153] An embodiment of the present application further provides an electronic device, including: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the above method for determining the evaluation index of the software project is executed when the program is run.

[0154] The processor is used to run a program that performs the following functions: determine the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores of the software project; determine the pre-evaluation index based on the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result scores; determine the overall project progress score, expected investment completion score, carry-over score, expected effectiveness achievement score, and overall user satisfaction score of the software project; determine the post-evaluation index based on the overall project progress score, expected investment completion score, carry-over score, expected effectiveness achievement score, and overall user satisfaction score; determine the target evaluation index of the software project based on the pre-evaluation index and the post-evaluation index.

[0155] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0156] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0157] In the above-mentioned embodiments of the present application, the collected information is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary protection measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0159] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0160] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0161] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0162] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining evaluation indicators of a software project, characterized in that: include: Determine the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result score of the software project; Determine the pre-evaluation index according to the investment change score, the expected investment completion rate score, the expected overall demand effectiveness score, and the pre-evaluation and post-evaluation result scores; Determine the overall project progress score, expected investment completion score, carry-over score, expected results achievement score, and overall user satisfaction score of the software project; Determine the post-evaluation indicators based on the overall project progress score, the expected investment completion score, the carry-over score, the expected results achievement score, and the overall user satisfaction score; According to the pre-evaluation index and the post-evaluation index, a target evaluation index of the software project is determined.

2. The method according to claim 1, characterized in that Determine the software project's investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result score, including: Determine the investment change score based on the overall investment change rate and the growth rate of the main business income of the current year, where the overall investment change rate is the difference between the overall planned investment amount of the current year and the overall planned investment amount of the previous year; Determine the expected investment completion rate score based on the expected investment completion rate for the current year, where the expected investment completion rate for the current year is the ratio of the expected annual account amount to the annual budget total; Determine the expected overall demand effectiveness score based on the number of expected overall demand effectiveness indicators actually reported that do not deviate from the plan and the number of expected overall demand effectiveness indicators proposed in the current plan; Determine the post-evaluation scores of the single project dimensions in the two years before this year, and take the weighted sum of the post-evaluation scores of the single project dimensions in the two years before this year and the post-evaluation scores of the overall project dimensions in the two years before this year to obtain the previous post-evaluation result scores.

3. The method according to claim 2, characterized in that Determine the evaluation scores of the single project dimensions in the first two years of this year, including: Determine the scores of the management standardization of the construction quality sub-indicator, the business support score of the construction quality sub-indicator, the capability openness score of the construction quality sub-indicator, and the technological advancement score of the construction quality sub-indicator; Performing weighted summation on the score of the management standardization of the construction quality sub-indicator, the score of the business support of the construction quality sub-indicator, the score of the capability opening of the construction quality sub-indicator, and the score of the technical advancement of the construction quality sub-indicator to obtain a first summation result; Determine the score of the application effect sub-indicator user satisfaction and the score of the application effect sub-indicator user activity, and perform weighted summation on the scores of the application effect sub-indicator user satisfaction and the scores of the application effect sub-indicator user activity to obtain a second summation result; Determine the score of the benefit sub-indicator input-output ratio and the score of the benefit sub-indicator demand effectiveness realization rate, and perform weighted summation on the score of the benefit sub-indicator input-output ratio and the score of the benefit sub-indicator demand effectiveness realization rate to obtain a third summation result; The first summation result, the second summation result and the third summation result are weightedly summed to obtain the post-evaluation scores of the single-project dimension for the two years before the current year.

4. The method according to claim 3, characterized in that Determine the scores of the management standardization of the construction quality sub-indicator, the business support score of the construction quality sub-indicator, the capability openness score of the construction quality sub-indicator, and the technological advancement score of the construction quality sub-indicator, including: Determine the score of the management standardization of the construction quality sub-indicator according to whether the actual construction period in the project demand freezing and handover stage is less than the standard construction period, whether the actual construction period in the project feasibility study stage is less than the standard construction period, whether the actual construction period in the project design stage is less than the standard construction period, whether the actual construction period in the project construction stage is less than the standard construction period, whether the actual construction period in the project initial inspection stage is less than the standard construction period, whether the actual construction period in the project trial operation stage is consistent with the standard construction period, and whether the actual construction period in the project final inspection stage is less than the standard construction period; Determine the score of the business support of the construction quality sub-indicator based on the number of realized business requirements and the total number of business requirements proposed in the project; The score of the capacity opening of the construction quality sub-indicator is determined based on the ratio of the number of open capabilities actually used in the post-evaluation stage to the number of open capabilities reported in the project establishment stage and whether there are new open capabilities in the current project; The score of the technical advancement of the construction quality sub-indicator is determined based on whether the project has a promotion effect inside and outside the enterprise after it is launched, whether the construction of this project generates invention patents, whether the construction of this project generates enterprise standards, whether the construction of this project generates industry standards, whether the construction of this project generates national standards, whether the construction of this project generates enterprise-level awards, whether the construction of this project generates provincial and ministerial awards, and whether the construction of this project wins national awards.

5. The method according to claim 3, characterized in that: Determine the scores of the user satisfaction and user activity sub-indicators of the application effect, including: Determine the score of the user satisfaction of the application effect sub-indicator according to the number of user satisfaction survey results with 0 stars, the number of user satisfaction survey results with 1 star, the number of user satisfaction survey results with 2 stars, the number of user satisfaction survey results with 3 stars, the number of user satisfaction survey results with 4 stars, and the number of user satisfaction survey results with 5 stars; Determine the score of the user activity of the application effect sub-indicator according to the number of valid registered users who have logged into the system in the past six months and the number of registered users of the system; Determine the score of the benefit sub-indicator input-output ratio and the score of the benefit sub-indicator demand effectiveness realization rate, including: Determine the score of the input-output ratio of the benefit sub-indicator based on the total income amount at the evaluation time, the total investment amount over the years, and whether the project can provide data sources and measurement processes; The score of the realization rate of the benefit sub-indicator's demand effectiveness is determined based on the number of achieved results in the post-evaluation stage and the total number of expected demand results proposed in the pre-evaluation stage.

6. The method according to claim 1, characterized in that Determine the overall project progress score, expected investment completion score, carry-over score, expected results achievement score, and overall user satisfaction score of the software project, including: Determine the overall project progress score based on the number of projects whose overall construction period does not exceed the standard construction period after the post-evaluation and the number of projects participating in the post-evaluation; Determine the score of the expected investment completion based on the actual annual account amount, the expected investment completion rate for the current year and the total annual budget; The score of the carry-over situation is determined by dividing the overall carry-over amount for the year by the total investment amount at the time of the year; Determine the score of the expected results based on the number of achieved results in the overall post-evaluation phase and the total number of expected results in the overall pre-evaluation phase; The overall user satisfaction score is determined according to the number of projects in the pre-set branch whose user satisfaction index scores of the post-evaluation projects are greater than or equal to the number of projects in the pre-evaluation and the number of projects in the post-evaluation.

7. The method according to claim 1, characterized in that After determining the target evaluation index of the software project, the method further includes: Evaluate the software project according to the target evaluation index to obtain an evaluation result, wherein the evaluation result includes: excellent, good, pass or fail; For software projects with excellent evaluation results, increase their investment amount in the planned total investment stage; For software projects whose evaluation results are unsatisfactory, the investment amount in the planned total investment stage is reduced.

8. A device for determining evaluation indicators of a software project, characterized in that: include: The first determination module is used to determine the investment change score, expected investment completion rate score, expected overall demand effectiveness score, and pre- and post-evaluation result score of the software project; The second determination module is used to determine the pre-evaluation index according to the investment change score, the expected investment completion rate score, the expected overall demand effectiveness score, and the pre-evaluation result score; The third determination module is used to determine the overall project progress score, the expected investment completion score, the carry-over score, the expected results score, and the overall user satisfaction score of the software project; A fourth determination module is used to determine the post-evaluation index according to the overall project progress score, the expected investment completion score, the carry-over score, the expected effect achievement score, and the overall user satisfaction score; The fifth determination module is used to determine the target evaluation index of the software project according to the pre-evaluation index and the post-evaluation index.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the method for determining the evaluation index of the software project according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program, when running, executes the method for determining the evaluation index of a software project according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for determining the evaluation index of a software project according to any one of claims 1 to 7 is implemented.

Images