Multi-dimensional software project quality evaluation method and system, electronic equipment and storage medium

Abstract

The invention provides a multi-dimensional software project quality evaluation method which comprises the following steps: acquiring full life cycle data of a software project, and performing multi-source data fusion after unified cleaning conversion; inputting the fused data into a pre-constructed multi-dimensional quality evaluation model, dynamically adjusting the weight value of each dimension according to the project characteristics based on a pre-trained LSTM network, and summing the weights to generate a project quality comprehensive score; the dimensions comprise flow, quality, user experience, efficiency and user feedback; and automatically generating a risk matrix through a pre-trained NLP model according to the comprehensive scoring result of the project quality, improving suggestions and predicting the project quality. According to the technical scheme, the problems of hysteresis quality of project quality evaluation, single one-sidedness of evaluation and mismatching in a time-sensitive weight adjustment solution stage are effectively solved, accurate improvement measures can be given, and the project development efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software project evaluation, and in particular to a multi-dimensional software project quality evaluation method, system, electronic device and storage medium. TECHNICAL BACKGROUND

[0002] With the continuous improvement of software system complexity and the wide application of agile development, the traditional software quality evaluation method is facing severe challenges. The existing quality evaluation system has three core problems, one is the single evaluation dimension. The current mainstream tools (such as SonarQube, JaCoCo, etc.) mainly rely on limited indicators such as code coverage and defect quantity, which leads to the evaluation blind area of "high coverage and low quality". The second is the lack of dynamic adaptability. In the continuous delivery environment, the traditional stage evaluation (such as milestone review) has serious lag. The third is the insufficient data value mining. There is a serious data island between project management tools (such as Zen) and user feedback systems (such as meterSphere), which cannot map user feedback data to specific test cases.

[0003] TECHNICAL CONTENT

[0004] The present application aims to solve at least one of the above technical problems.

[0005] To solve the above problems, the present application provides a multi-dimensional software project quality evaluation method, device, electronic device and storage medium.

[0006] In the first aspect, the present application provides a multi-dimensional software project quality evaluation method, comprising:

[0007] Obtain software project full life cycle data, and perform multi-source data fusion after unified cleaning and conversion;

[0008] Input the fused data into a pre-constructed multi-dimensional quality evaluation model, dynamically adjust the weight values of each dimension based on the pre-trained LSTM network and according to the project characteristics, and generate a project quality comprehensive score after weight addition; the dimensions include process, quality, user experience, performance and user feedback;

[0009] According to the project quality comprehensive score result, a risk matrix, improvement suggestion and project quality prediction are automatically generated through a pre-trained NLP model.

[0010] Optionally, the multi-dimensional quality evaluation model comprises: an index layer, which standardizes specific indexes under each dimension, divides the indexes into positive indexes and negative indexes according to the index properties, and maps the indexes to a preset score interval after being processed by different normalization methods; a dimension layer, which adopts an attention mechanism to calculate the weights of indexes in each dimension, and automatically reduces the redundant weights of highly correlated indexes; and a comprehensive layer, which analyzes the time sequence characteristics of the scores of each dimension by using the LSTM network, and outputs dynamic weight coefficients of each dimension.

[0011] Optionally, the pre-trained LSTM network adopts a double-layer structure design, the first layer of 64 neurons captures the short-term fluctuation characteristics of an index, and the second layer of 32 neurons identifies the long-term trend changes of each index under the dimension; and the pre-trained LSTM network adopts an incremental learning method to adapt to new project characteristics.

[0012] Optionally, the dynamic adjustment of the weight values of each dimension based on the pre-trained LSTM network and according to the project characteristics comprises:

[0013] constructing a multi-dimensional project feature vector according to project characteristics; wherein the project feature dimensions include an industry type, a system key level, and a development mode, the industry type is encoded by using a one-hot encoding, the system key level is quantified by using a numerical value, and the development mode is numerically mapped to different methodologies;

[0014] designing a weight bias injection module after the output of the standard LSTM network; the module comprises a trainable bias matrix, the rows of the matrix correspond to five dimensions, and the columns of the matrix correspond to the project feature dimensions;

[0015] when the feature vector of a specific type of project is input, the weight adjustment amount that each dimension should obtain is calculated by matrix multiplication.

[0016] Optionally, the dynamic adjustment of the weight values of each dimension based on the pre-trained LSTM network and according to the project characteristics further comprises: designing a time decay function w_t = w_0 x e^(-λt) + Δw_LSTM after the output of the standard LSTM network; wherein w_0 is the initial weight of the historical template, λ is an industry decay coefficient, and Δw_LSTM is the dynamic adjustment amount output by the model.

[0017] Optionally, the method further comprises establishing a typical project historical weight library, if the similarity between the current project characteristics and the historical project characteristics is greater than 70%, automatically loading the weight of the historical project at the same stage as the initial value of the LSTM network, and setting a dynamic adjustment range limit.

[0018] Optionally, the risk matrix divides risk levels and matches treatment measures according to the score and the trend; the improvement suggestion automatically warns and generates an improvement scheme when any dimension score is lower than a threshold; and the project quality prediction includes comparison of the trend of each dimension score and a key indicator.

[0019] In a second aspect, the present application provides a multi-dimensional software project quality evaluation system, comprising:

[0020] A data acquisition module is configured to acquire software project lifecycle data.

[0021] A multi-source data fusion engine is configured to perform multi-source data fusion after unified cleaning and conversion of the acquired data.

[0022] A scoring model engine comprises a pre-constructed multi-dimensional quality evaluation model, configured to input the fused data into the multi-dimensional quality evaluation model, dynamically adjust the weight values of each dimension based on a pre-trained LSTM network and according to project characteristics, and generate a project quality comprehensive score after weight addition; the dimensions include process, quality, user experience, performance, and user feedback.

[0023] A visualization and feedback module is configured to automatically generate a risk assessment report, an improvement suggestion, and a project quality prediction based on the project quality comprehensive score result through a pre-trained NLP model.

[0024] In a third aspect, the present application provides an electronic device comprising a memory and a processor.

[0025] The memory is configured to store a computer program.

[0026] The processor is configured to implement the multi-dimensional software project quality evaluation method of the first aspect when executing the computer program.

[0027] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the multi-dimensional software project quality evaluation method of the first aspect.

[0028] The technical solution of the present application evaluates the overall quality of a project from five dimensions and the whole lifecycle, and evaluates the overall project at different stages according to different characteristics: different dimensions and different indicators under different dimensions have different weight values dynamically adjusted by LSTM, so that the evaluation score is more comprehensive and convincing. Moreover, the data after evaluation is combined with an NLP model to give corresponding risks and suggestions, which can quickly and in advance identify problems and risks and make corresponding adjustments during the project development and testing stage, so as to ensure that the project is completed on schedule and with quality.

[0029] The accompanying drawings

[0030] Figure 1 A flowchart of a multi-dimension software project quality evaluation method of an embodiment of the present application;

[0031] Figure 2 A principle diagram of dynamic weight adjustment of each dimension in an embodiment of the present application;

[0032] Figure 3 A principle diagram of a multi-dimension software project quality evaluation system of an embodiment of the present application;

[0033] Figure 4 A principle diagram of a visualization and feedback module in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the above objectives, features and advantages of the present application more apparent, specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, rather, these embodiments are provided to make the present application more thorough and complete. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes, and are not intended to limit the scope of protection of the present application.

[0035] It should be understood that each step described in the method embodiments of the present application can be executed in different orders, and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present application is not limited in this respect.

[0036] The term "comprising" and variations thereof as used in the present application are open-ended, that is, "including but not limited to"; the term "based on" is "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". Related definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in the present application are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0037] It should be noted that the modification of "one" or "multiple" mentioned in the present application is illustrative rather than limiting, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0038] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present application are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0039] As shown in Figure 1 and Figure 2 The multi-dimensional software project quality evaluation method provided by the embodiment of the present application realizes quantitative evaluation from micro indicators to macro quality through a multi-level scoring architecture; adopts a three-level scoring model of "indicator layer-dimension layer-comprehensive layer", combines LSTM dynamic weight adjustment and NLP sentiment analysis, and realizes accurate prediction of project quality, risk warning and suggestion. The specific scheme is as follows:

[0040] Step S1, acquire software project full life cycle data, and perform multi-source data fusion after unified cleaning and conversion.

[0041] Among them, the data source in the embodiment mainly interfaces with three systems: Chan Dao project management tool, MeterSphere test platform and user feedback system. Data is collected through standardized interface protocol, Chan Dao system mainly acquires requirement document version, task progress state and defect tracking record; test platform collects test case execution result, automation test coverage and performance index data; user feedback system collects terminal user score and text evaluation content. All data is cleaned and converted through a unified data pipeline, including null value filling, outlier correction and data format standardization, and finally stored in an analytical database, and the query efficiency is optimized according to project ID sharding and partitioning.

[0042] A cross-system data association mechanism is established, and Chan Dao and meterSphere are connected through project name and project id, and bug and problem tracking single of test failure record are associated through project id. The fused data entity contains complete requirement-case-defect-feedback link, and supports full-process traceability analysis.

[0043] Step S2, input the fused data into the pre-constructed multi-dimensional quality evaluation model, dynamically adjust the weight values of each dimension based on the pre-trained LSTM network and according to the project characteristics, and generate the project quality comprehensive score after weight addition; each dimension includes process, quality, user experience, performance and user feedback.

[0044] Specifically, the multi-dimensional quality evaluation model includes three levels:

[0045] (1) Indicator layer, 21 specific indicators are standardized, and according to the nature of the indicators, they are divided into positive indicators and negative indicators, different normalization methods are used respectively, and all indicator scores are mapped to the interval of 0-100. In this embodiment, the specific indicators and dimensions are divided into: the evaluation indicators of the process dimension (PRO) include: the number of requirement reviews, the number of technical scheme reviews, the delay of test submission, and the delay of release delivery; the evaluation indicators of the quality dimension (QA) include: defect severity distribution, defect quantity trend, defect activation rate, and smoke test pass rate; the evaluation indicators of the user experience dimension (UED) include: interface and document quality, system usability score, and interaction problem quantity; the evaluation indicators of the efficiency dimension (EFF) include: automation test coverage rate, continuous integration execution efficiency, and code review pass rate; and the evaluation indicators of the user feedback dimension (UF) include: user satisfaction score, usage problem feedback quantity, and function requirement satisfaction degree.

[0046] (2) Dimension layer, the attention mechanism is used to calculate the weight of each dimension internal indicator, the influence of the correlation between indicators is considered, and the redundant weight of the highly correlated indicators is automatically reduced.

[0047] (3) Comprehensive layer, the time sequence characteristics of the five-dimensional scores are analyzed through an LSTM network, and the contribution weight of each dimension in the overall score is dynamically adjusted; the input of the LSTM network is a 15-day historical score sliding window, and the output is the dynamic weight coefficient of each dimension.

[0048] Particularly, the LSTM network in the technical solution adopts a double-layer structure design, the first layer of 64 neurons captures the short-term fluctuation characteristics of an indicator, and the second layer of 32 neurons identifies the long-term trend change of each indicator under the dimension. The AI training data thereof contains the data dimension scores and final quality labels of historical iterations in the project, and the loss function adds a weight smoothing constraint to avoid sharp fluctuations. The trained LSTM network model is automatically updated every week, and an incremental learning method is used to adapt to new project characteristics.

[0049] In the dynamic weight adjustment scheme of the multi-dimensional quality evaluation model of the application, a multi-dimensional project feature vector is constructed according to the dynamic weight of the project characteristics, including three types of characteristics: industry type, system key level and development mode. The industry type adopts one-hot encoding, such as distinguishing the main fields of finance, medical treatment and Internet of Things; each system key function or process level is quantified by a numerical value, the core system is marked as 1.0, the important system is 0.7, and the general system is 0.3; and the development mode is numerically mapped to different methodologies such as agile and waterfall. These characteristics together form a feature vector as a digital representation of project characteristics.

[0050] After the standard LSTM network output, a weight bias injection module is designed. This module contains a trainable bias matrix, with rows corresponding to the five evaluation dimensions and columns corresponding to the project feature dimensions. When a specific type of project, such as a financial system, is input with a feature vector, the weight adjustment amount for each dimension is calculated through matrix multiplication. Among them, financial projects will obtain significant positive bias in the QA dimension, and medical projects will be strengthened in the UED dimension. The bias is normalized by Softmax to ensure that the sum of the maximum weights is 1.

[0051] Preferably, the multi-dimensional quality evaluation model of the present application can realize historical weight backtracking according to the preset historical weight library. For example, when the current project feature matches the similarity of the historical project > 70%, load the historical project weight at the same stage as the LSTM initial value, and allow dynamic adjustment range ± 30%. In particular, the financial project limit can be set to ± 15%. The preset historical weight library in the present embodiment includes: (1) Financial project template: set QA basic weight to 0.35 (industry compliance requirement); (2) Medical project template: set UED basic weight to 0.3 (user experience sensitive); (3) Government project template: set PRO basic weight to 0.4 (process compliance priority).

[0052] For easy understanding, the application scenario of a certain e-commerce promotion project in the present embodiment is exemplified as follows:

[0053] According to the dimensional time variation of Table 1, the dynamic adjustment process is: "When the defect density in the test phase is monitored to be > 5 per thousand lines of code, the QA dimension weight is automatically increased by 10%. At the same time, trigger: requirement review index weight decreases by 5% (historical data shows that emergency repair is better than process improvement); The automation test coverage rate threshold is increased from 80% to 85%."

[0054] Phase PRO QA UED EFF UF Requirement phase 0.35 0.25 0.15 0.15 0.1 Test phase 0.2 0.4 0.1 0.25 0.05 3 days after going live 0.15 0.3 0.2 0.15 0.2

[0055] Table 1 changes with time

[0056] Preferably, the multi-dimensional quality evaluation model of the present application also introduces a time decay factor, and each dimension weight automatically evolves with the project stage. An implementable solution provided in the present embodiment is as follows:

[0057] "Requirement phase (0-20% progress): PRO dimension weight accounts for 35% (demand review times weight increase);

[0058] Development phase (20-50% progress): QA dimension weight increases to 40% (defect trend weight doubles);

[0059] Test phase (50-80% progress): EFF dimension weight increases by 30% (automation coverage rate weight peak);

[0060] Post-release phase (80-100% progress): Linear growth of UF dimension weight (5% increase per week).

[0061] For ease of understanding, an actual accounting system scenario is cited as an example:

[0062] "Input features: Financial industry label + core system label

[0063] Weight change: QA dimension: 0.22→0.31 (+41%); EFF dimension: 0.25→0.21 (-16%)

[0064] Time decay function design: w_t = w_0 × e^{-λt} + Δw_{LSTM}, where: w_0: initial weight of historical template; λ: industry decay coefficient (λ=0.1 for financial industry, λ=0.3 for internet industry); Δw_{LSTM}: dynamic adjustment amount output by the model.

[0065] Step S3, according to the project quality comprehensive score result, automatically generating risk matrix, improvement suggestion and project quality prediction through pre-trained NLP model.

[0066] Referring to the accompanying Figure 4 , the real-time risk matrix divides the risk level and matches the disposal measures according to the score and the trend; the improvement suggestion automatically warns when any dimension score is lower than the threshold value, and provides an optimization scheme based on the historical similar projects in the knowledge base and the training using NLP; the project quality prediction displays the dimension score trend and the key index comparison through the visual board. The report generation adopts a hierarchical strategy, generates a complete analysis report every day, and triggers an early warning notification in real time for key abnormalities. The analysis and suggestion module is displayed in green, and the risk and warning is displayed in red.

[0067] As shown in Figure 3 , the embodiment of the present application provides a multi-dimensional software project quality evaluation system, comprising:

[0068] A data acquisition module is configured to acquire software project lifecycle data.

[0069] A multi-source data fusion engine is configured to perform multi-source data fusion after unified cleaning and conversion of the acquired data.

[0070] A scoring model engine includes a pre-built multi-dimensional quality evaluation model, configured to input the fused data into the multi-dimensional quality evaluation model, dynamically adjust the weight values of each dimension based on a pre-trained LSTM network and according to the project characteristics, and generate a comprehensive project quality score after weight addition; the dimensions include process, quality, user experience, efficiency and user feedback.

[0071] A visualization and feedback module is configured to automatically generate a risk assessment report, improvement suggestions, and a project quality prediction based on the project quality comprehensive score result by using a pre-trained NLP model.

[0072] The embodiment of the present application also provides an electronic device, including a memory and a processor; the memory is used for storing a computer program; the processor is used for implementing the multi-dimensional software project quality evaluation method when the computer program is executed.

[0073] The embodiment of the present application provides a computer readable storage medium, and the storage medium stores a computer program, and when the computer program is executed by a processor, the multi-dimensional software project quality evaluation method is implemented.

[0074] An electronic device that can be a server or a client of the present application will now be described, which is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent a wide variety of digital electronic computing devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device can also represent a variety of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections, and their functions, as well as their interrelationships, are merely examples and are not intended to limit implementations of the present application described and / or claimed herein.

[0075] The electronic device includes a computing unit that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The computing unit, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0076] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), or the like. In this application, the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present application. In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0077] Although the present application is disclosed as above, the protection scope of the present application is not limited to this. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and these changes and modifications will fall within the protection scope of the present application.

Claims

1. A multi-dimensional software project quality evaluation method, characterized in that... include: Acquire data from the entire lifecycle of software projects, clean and transform it uniformly, and then perform multi-source data fusion. The fused data is input into a pre-constructed multi-dimensional quality assessment model. Based on a pre-trained LSTM network, the weights of each dimension are dynamically adjusted according to the characteristics of the project. The weights are summed to generate a comprehensive project quality score. The dimensions include process, quality, user experience, efficiency, and user feedback. Based on the comprehensive project quality score, a risk matrix, improvement suggestions, and project quality predictions are automatically generated using a pre-trained NLP model.

2. The multi-dimensional software project quality evaluation method as described in claim 1, characterized in that... The multi-dimensional quality assessment model includes: The indicator layer standardizes the specific indicators under each dimension, classifies them into positive and negative indicators according to their nature, and maps them to a preset score range after processing them with different normalization methods. At the dimensional layer, an attention mechanism is used to calculate the weights of indicators within each dimension, automatically reducing redundant weights of highly correlated indicators. The synthesis layer analyzes the temporal characteristics of the scores of each dimension through the LSTM network and outputs the dynamic weight coefficients of each dimension.

3. The multi-dimensional software project quality evaluation method as described in claim 1, characterized in that... The pre-trained LSTM network adopts a two-layer structure design. The first layer of 64 neurons captures the short-term fluctuation characteristics of a certain indicator, and the second layer of 32 neurons identifies the long-term trend changes of each indicator under the specified dimension. The pre-trained LSTM network adopts an incremental learning method to adapt to the characteristics of new projects.

4. A multi-dimensional software project quality evaluation method as described in any one of claims 1-3, characterized in that... The pre-trained LSTM network and dynamic adjustment of weights for each dimension based on project characteristics include: A multi-dimensional project feature vector is constructed based on the project characteristics. The project feature dimensions include industry type, system criticality level, and development mode. The industry type adopts one-hot encoding, the system criticality level is quantified numerically, and the development mode is numerically mapped to different methodologies. After the output of the standard LSTM network, a weight bias injection module is designed; the module includes a trainable bias matrix, where the matrix rows correspond to the five dimensions and the matrix columns correspond to the item feature dimensions. When a feature vector of a specific type of project is input, the weight adjustment amount that should be obtained for each dimension is calculated by matrix multiplication.

5. The multi-dimensional software project quality evaluation method as described in claim 4, characterized in that... The pre-trained LSTM network and dynamic adjustment of weights for each dimension based on project characteristics also include: After the output of the standard LSTM network, the time decay function is designed as: w_t=w_0×e^{-λt}+Δw_{LSTM}; where w_0: initial weights of the historical template; λ: industry decay coefficient; Δw_{LSTM}: dynamic adjustment of the model output.

6. The multi-dimensional software project quality evaluation method as described in claim 1, characterized in that... It also includes establishing a historical weight library for typical projects. If the similarity between the current project features and historical projects is greater than 70%, the weights of the historical projects in the same stage are automatically loaded as the initial values ​​of the LSTM network, and a dynamic adjustment range limit is set.

7. The multi-dimensional software project quality evaluation method as described in claim 1, characterized in that... The risk matrix classifies risk levels and matches corresponding measures based on scores and trends; the improvement suggestions automatically issue warnings and generate improvement plans when the score of any dimension falls below a threshold; the project quality prediction includes a comparison of score trends and key indicators for each dimension.

8. A multi-dimensional software project quality evaluation system, characterized in that... include: The data acquisition module is used to acquire data throughout the entire lifecycle of a software project; A multi-source data fusion engine is used to clean and transform the collected data before fusing it from multiple sources. The scoring model engine includes a pre-built multi-dimensional quality assessment model, which is used to input the fused data into the multi-dimensional quality assessment model, and dynamically adjust the weight values ​​of each dimension based on a pre-trained LSTM network according to the characteristics of the project. The weights are summed to generate a comprehensive project quality score. The dimensions include process, quality, user experience, efficiency, and user feedback. The visualization and feedback module is used to automatically generate risk assessment reports, improvement suggestions, and project quality predictions based on the comprehensive project quality score results using a pre-trained NLP model.

9. An electronic device, characterized in that... Including memory and processor; The memory is used to store computer programs; The processor is configured to implement, when executing the computer program, a multi-dimensional software project quality evaluation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements a multi-dimensional software project quality evaluation method as described in any one of claims 1 to 7.

Images