Power grid dispatching management software quality evaluation method and system based on analytic hierarchy process

Abstract

The invention discloses a power grid dispatching management software quality evaluation method and system based on an analytic hierarchy process. The method comprises the following steps: constructing a multi-level quality analysis model based on document data of different stages in a software cycle; for the multilevel quality analysis model, performing expert scoring based on evaluation indexes, constructing a judgment matrix, and performing consistency check; the method comprises the following steps: extracting an abstract of a document on the basis of a Randeng-Pegasus-523M-Chine model, automatically extracting keywords in the document on the basis of a TextRank algorithm, and quickly evaluating the software related document; and performing index evaluation on the software quality. According to the method, the software quality is comprehensively evaluated by analyzing documents of all stages in the life cycle, and the documents comprise a demand document, a design document, a development document, a test report, user feedback and the like which record the whole process of software from birth to maturity; by deeply analyzing the documents, the performance of the software on various quality indexes, such as functionality, reliability, usability, maintainability and the like, can be more comprehensively and meticulously understood.

Description

Technical Field

[0001] The present invention relates to the technical field of software quality assessment, and in particular to a method and system for assessing the quality of power grid dispatching management software based on a hierarchical analysis method. Background Art

[0002] Grid dispatching automation software is one of the key technologies for grid operation management. It is mainly used to realize the functions of real-time monitoring, analysis, prediction, optimized dispatching and fault handling of grid operation. As the core tool for grid dispatching control, grid dispatching software undertakes the important responsibilities of real-time monitoring of grid status, accurate analysis of operation data, intelligent formulation of dispatching plans and rapid response to grid changes. Its quality is directly related to the stable operation, efficiency improvement and safety and reliability of the grid. High-quality grid dispatching software can ensure the stable operation of the grid. Through accurate data analysis and intelligent dispatching strategies, it can effectively prevent the occurrence of grid failures, reduce power outages, and ensure the continuity and reliability of power supply. At the same time, it can also improve the operation efficiency of the grid, realize the economic operation of the grid by optimizing resource allocation and reducing energy consumption, and bring practical economic benefits to power companies and users. Software quality evaluation is a comprehensive, objective and scientific evaluation of software products to ensure that the software can meet the needs and expectations of users and has good performance, reliability, ease of use, security and other characteristics. In the process of software quality evaluation, a series of testing methods and techniques are usually used, such as unit testing, integration testing, system testing, acceptance testing, etc., to ensure that the software quality meets the predetermined standards and requirements. At the same time, we will continue to improve and optimize the software based on user feedback and market demand to enhance its quality and competitiveness.

[0003] Commonly used software evaluation methods include analytic hierarchy process, fuzzy comprehensive evaluation method, Bayesian network method, neural network method, etc. Fuzzy analytic hierarchy process (FAHP) is an improvement on the traditional analytic hierarchy process (AHP), which takes into account the fuzziness and uncertainty in the decision-making process. AHP is a decision analysis method that combines qualitative and quantitative methods. It decomposes complex problems into a hierarchical structure and uses pairwise comparison and expert scoring to determine the relative importance and weight of each factor. For software quality evaluation, existing methods usually use analytic hierarchy process to construct a product quality hierarchical model, starting from factors closely related to quality such as software reliability, usability, security, compatibility, performance efficiency, etc., and calculating the evaluation values ​​of different factors and the evaluation value of software quality based on criteria and weight vectors.

[0004] Software quality evaluation involves multiple dimensions. Existing methods focus on evaluating the quality of the software itself, and evaluate the key elements and criteria of the software through unit testing, integration testing, system testing, etc. Grid dispatching software is the core tool of the power grid, and has high requirements for accuracy and stability. Its quality is directly related to the stable operation, efficiency improvement, safety and reliability of the power grid, and is the cornerstone of the normal operation of modern society. Efficient and accurate evaluation of grid dispatching software is an important prerequisite for ensuring its stable operation. For the quality of software, in addition to test indicators and the main evaluation of the software itself, the entire life cycle of the software has a significant impact on the quality of the software, including from the initial requirements, design, development, testing and operation and maintenance.

[0005] Traditional software quality assessment methods, such as testing and trial runs, can directly assess the performance and functionality of the software itself, but they are often limited to the current version or status of the software and it is difficult to fully grasp the performance of the software throughout its life cycle. This limitation may cause some potential problems to be overlooked, thus affecting the long-term stability of the software and user experience. Summary of the invention

[0006] Purpose of the invention: The present invention provides a method and system for evaluating the quality of power grid dispatching management software based on the hierarchical analysis method, which can improve the efficiency and accuracy of the quality evaluation of power grid management and dispatching software.

[0007] Technical solution: The method for evaluating the quality of power grid dispatching management software based on the analytic hierarchy process described in the present invention comprises the following steps:

[0008] Step 1: Construct a multi-level quality analysis model based on the documentation of different stages in the software life cycle;

[0009] Step 2: For the multi-level quality analysis model in step 1, experts score based on the evaluation indicators, construct a judgment matrix, and perform consistency test;

[0010] Step 3: Extract the document summary based on the Randeng-Pegasus-523M-Chinese model, automatically extract keywords from the document based on the TextRank algorithm, and quickly evaluate the software-related documents;

[0011] Step 4: Evaluate the software quality indicators; based on the results of step 3, evaluate the quality of the software process related documents one by one, and apply the judgment matrix based on step 2 to the document evaluation results to obtain the evaluation indicators of the comprehensive quality of the software.

[0012] Furthermore, in step 1, the multi-level quality analysis model adopts the waterfall model, which divides the entire software cycle into six stages: software planning, requirement analysis, software design, program coding, software testing, and operation and maintenance; the software planning stage includes a feasibility study report and a project development plan; the requirement analysis stage includes business requirement documents, market requirement documents, user requirement documents, functional requirement specifications, non-functional requirement specifications, requirement specifications, use cases and scenario documents, interface prototypes and designs, data dictionaries, and requirement tracking matrices; the software design stage includes system design documents, architecture design documents, component design documents, interface design documents, data design documents, use case implementation documents, design decision documents, and design specification documents; the software testing stage includes test plan documents, test case documents, test script documents, test matrix documents, test execution reports, defect management documents, test summary documents, and regression test documents; the operation and maintenance stage includes user manuals, maintenance plans, troubleshooting guides, system monitoring reports, change management records, security management documents, and business continuity plans.

[0013] Furthermore, in step 2, experts score based on the evaluation indicators, and the judgment matrix is ​​constructed as follows: The meaning of the numerical scale of the pairwise comparison scores is listed in Table 1:

[0014] Table 1 Meaning of digital scale

[0015]

[0016] Based on the expert scoring data, a judgment matrix is ​​generated:

[0017]

[0018] Furthermore, in step 2, the consistency check includes a hierarchical single-order consistency check and a hierarchical total-order consistency check.

[0019] Furthermore, the hierarchical single sorting consistency test is as follows:

[0020] Calculate the consistency index CI:

[0021]

[0022] Find and calculate the average random consistency index RI, as listed in Table 2:

[0023] Table 2 Average random consistency index RI standard value

[0024]

[0025]

[0026] Finally, calculate the consistency ratio CR:

[0027]

[0028] When CR<0.1, the consistency of the judgment matrix is ​​acceptable. Otherwise, the judgment matrix is ​​appropriately adjusted and corrected. After the correction, the consistency test is performed again until the consistency test passes.

[0029] Furthermore, the hierarchical total ranking consistency test is as follows: hierarchical total ranking is the process of determining the ranking weights of all factors in a certain layer for the overall goal. If the previous layer A contains n factors B1, B2, ..., B n , whose hierarchical weight values ​​are b1, b2, ..., b n , if some factors of the next level C are important for B # The consistency index of a single sort is CI # (i=1,2,3……n), the corresponding average random consistency index is RI # , then the total random consistency ratio of the C level is:

[0030]

[0031] When CR<0.1, it is considered that the hierarchical total ranking results have satisfactory consistency and the analysis results are acceptable.

[0032] Furthermore, in step 3, the summary of the document is extracted based on the Randeng-Pegasus-523M-Chinese model as follows: large-scale Chinese text data is collected, including news articles, encyclopedia entries, social media content and other types of text to ensure that the model can learn rich language patterns and semantic information; in the pre-training stage, some sentences are deleted from the input text through training through the GSG task, and then the model is asked to predict these deleted sentences. This task simulates the summary generation process, so that the model learns how to generate sentences consistent with the context in the pre-training stage; a software process document summary dataset is constructed to fine-tune the model, and the model parameters are optimized through supervised learning to make it perform best in the summary generation task; according to the quality of the generated summary, the model is evaluated using the ROUGE score, and the model parameters and training strategy are adjusted according to the results.

[0033] Furthermore, in step 3, automatically extracting keywords from the document based on the TextRank algorithm specifically includes the following steps:

[0034] Step 31: perform data preprocessing operations such as word segmentation, part-of-speech tagging, and stop word removal on the text;

[0035] Step 32: construct a candidate keyword graph G = (V, E), where V is a node set (consisting of candidate keywords), and use the co-occurrence relationship to construct an edge between any two nodes. An edge exists between two nodes only if their corresponding words co-occur in a window of length K, where K represents the window size, i.e., a maximum of K co-occurring words;

[0036] Step 33, iteratively calculate the weight of each node until convergence;

[0037] Step 34: Arrange the node weights in reverse order to obtain the top N words as text keywords.

[0038] Further, in step 4, based on the extracted summary and keyword information, the documents of different stages (planning, requirements, design, testing, operation and maintenance) of the software process are quickly evaluated, and the applicability, completeness, accuracy, practicality, clarity, consistency, comprehensibility, maintainability, compliance, feasibility, correctness, risk, and flexibility index values ​​corresponding to each software development stage of the lowest layer (fourth layer) are counted; the evaluation value of the software quality of each stage of the third layer is calculated using the index value of the fourth layer and the weight matrix between the fourth layer and the third layer; then the evaluation value of the functionality, performance, reliability, security, ease of use, and maintainability of the second layer is calculated using the calculated third layer evaluation value and the weight matrix between the second layer and the third layer; finally, the overall evaluation value of the software quality is calculated using the weight matrix between the first layer and the second layer; for each evaluation, a consistent score range is established, including applicability, completeness, accuracy, practicality, clarity, consistency, comprehensibility, maintainability, compliance, feasibility, correctness, risk, and flexibility, and the scores of the evaluation indicators are taken as shown in the following table:

[0039] Table 3 Evaluation index value standards

[0040]

[0041] Accordingly, a power grid dispatching management software quality assessment system based on hierarchical analysis method includes: a model construction module, a consistency verification module, a document extraction module and a software quality assessment module; the model construction module constructs a multi-level quality analysis model based on the document data of different stages in the software cycle; the consistency verification module performs expert scoring based on the evaluation index, constructs a judgment matrix, and performs consistency verification; the document extraction module extracts the abstract of the document based on the Randeng-Pegasus-523M-Chinese model, automatically extracts keywords in the document based on the TextRank algorithm, and quickly evaluates the software-related documents; the software quality assessment module realizes the evaluation of software quality.

[0042] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: by analyzing the documents at each stage of the life cycle, the software quality is comprehensively evaluated. These documents include requirement documents, design documents, development documents, test reports, user feedback, etc., which record the entire process from the birth to the maturity of the software; by in-depth analysis of these documents, a more comprehensive and detailed understanding of the performance of the software in various quality indicators, such as functionality, reliability, ease of use, maintainability, etc., can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the method flow of the present invention.

[0044] Figure 2 It is a schematic diagram of the hierarchical model of the present invention. DETAILED DESCRIPTION

[0045] like Figure 1 and Figure 2 As shown, a method for evaluating the quality of power grid dispatching management software based on hierarchical analysis method includes the following steps:

[0046] Step 1: Build a multi-level quality analysis model based on the documentation at different stages of the software life cycle.

[0047] The multi-level quality analysis model adopts the waterfall model, which divides the entire software cycle into six stages: software planning, requirement analysis, software design, program coding, software testing, and operation and maintenance; the software planning stage includes a feasibility study report and a project development plan; the requirement analysis stage includes business requirement documents, market requirement documents, user requirement documents, functional requirement specifications, non-functional requirement specifications, requirement specifications, use cases and scenario documents, interface prototypes and designs, data dictionaries, and requirement tracking matrices; the software design stage includes system design documents, architecture design documents, component design documents, interface design documents, data design documents, use case implementation documents, design decision documents, and design specification documents; the software testing stage includes test plan documents, test case documents, test script documents, test matrix documents, test execution reports, defect management documents, test summary documents, and regression test documents; the operation and maintenance stage includes user manuals, maintenance plans, troubleshooting guides, system monitoring reports, change management records, security management documents, and business continuity plans.

[0048] Step 2: For the multi-level quality analysis model in step 1, experts score based on the evaluation indicators, construct a judgment matrix, and perform consistency test.

[0049] Experts scored based on the evaluation indicators, and constructed a judgment matrix as follows: The meaning of the numerical scale for pairwise comparison scoring is listed in Table 1:

[0050] Table 1 Meaning of digital scale

[0051]

[0052]

[0053] Based on the expert scoring data, a judgment matrix is ​​generated:

[0054]

[0055] The consistency check includes hierarchical single-order consistency check and hierarchical total-order consistency check.

[0056] The hierarchical single sorting consistency test is as follows:

[0057] Calculate the consistency index CI:

[0058]

[0059] Find and calculate the average random consistency index RI, as listed in Table 2:

[0060] Table 2 Average random consistency index RI standard value

[0061]

[0062] Finally, calculate the consistency ratio CR:

[0063]

[0064] When CR<0.1, the consistency of the judgment matrix is ​​acceptable. Otherwise, the judgment matrix is ​​appropriately adjusted and corrected. After the correction, the consistency test is performed again until the consistency test passes.

[0065] The hierarchical total ranking consistency test is as follows: hierarchical total ranking is the process of determining the ranking weights of all factors in a certain layer for the overall goal. If the previous layer A contains n factors B1, B2, ..., B n , whose hierarchical weight values ​​are b1, b2, ..., b n , if some factors of the next level C are important for B # The consistency index of a single sort is CI # (i=1,2,3……n), the corresponding average random consistency index is RI # , then the total random consistency ratio of the C level is:

[0066]

[0067] When CR<0.1, it is considered that the hierarchical total ranking results have satisfactory consistency and the analysis results are acceptable.

[0068] Step 3: Extract the document summary based on the Randeng-Pegasus-523M-Chinese model, automatically extract keywords from the document based on the TextRank algorithm, and quickly evaluate the software-related documents.

[0069] Collect large-scale Chinese text data, including news articles, encyclopedia entries, social media content and other types of text, to ensure that the model can learn rich language patterns and semantic information; in the pre-training stage, train through the GSG task, delete some sentences from the input text, and then let the model predict these deleted sentences. This task simulates the summary generation process, so that the model can learn how to generate sentences consistent with the context in the pre-training stage; build a software process document summary dataset to fine-tune the model, optimize the model parameters through supervised learning, and make it perform best in the summary generation task; evaluate the model based on the quality of the generated summary using the ROUGE score, and adjust the model parameters and training strategy based on the results.

[0070] Automatically extracting keywords from documents based on the TextRank algorithm specifically includes the following steps:

[0071] Step 31: perform data preprocessing operations such as word segmentation, part-of-speech tagging, and stop word removal on the text;

[0072] Step 32: construct a candidate keyword graph G = (V, E), where V is a node set (consisting of candidate keywords), and use the co-occurrence relationship to construct an edge between any two nodes. An edge exists between two nodes only if their corresponding words co-occur in a window of length K, where K represents the window size, i.e., a maximum of K co-occurring words;

[0073] Step 33, iteratively calculate the weight of each node until convergence;

[0074] Step 34: Arrange the node weights in reverse order to obtain the top N words as text keywords.

[0075] Rapid location and retrieval in large-scale system-related documents through abstracts and keywords can effectively improve the efficiency of document evaluation.

[0076] Step 4: Evaluate the software quality indicators. A consistent score range is established for each evaluation, and the scores of the evaluation indicators are determined as shown in the following table:

[0077] Table 3 Evaluation index value standards

[0078]

[0079]

[0080] Based on the extracted summary and keyword information, a rapid evaluation is conducted on the documents of different stages of the software process (planning, requirements, design, testing, operation and maintenance), and the applicability, completeness, accuracy, practicality, clarity, consistency, comprehensibility, maintainability, compliance, feasibility, correctness, risk, and flexibility index values ​​corresponding to each software development stage of the lowest layer (the fourth layer) are counted; the evaluation value of the software quality at each stage of the third layer is calculated using the index values ​​of the fourth layer and the weight matrix between the fourth layer and the third layer; then the evaluation values ​​of the functionality, performance, reliability, security, ease of use, and maintainability of the second layer are calculated using the calculated evaluation values ​​of the third layer and the weight matrix between the second layer and the third layer; finally, the overall evaluation value of the software quality is calculated using the weight matrix between the first layer and the second layer.

[0081] Accordingly, a power grid dispatching management software quality assessment system based on hierarchical analysis method includes: a model construction module, a consistency verification module, a document extraction module and a software quality assessment module; the model construction module constructs a multi-level quality analysis model based on the document data of different stages in the software cycle; the consistency verification module performs expert scoring based on the evaluation index, constructs a judgment matrix, and performs consistency verification; the document extraction module extracts the abstract of the document based on the Randeng-Pegasus-523M-Chinese model, automatically extracts keywords in the document based on the TextRank algorithm, and quickly evaluates the software-related documents; the software quality assessment module realizes the evaluation of software quality.

[0082] The present invention comprehensively evaluates the software quality by analyzing the documents at each stage of the life cycle. These documents include requirement documents, design documents, development documents, test reports, user feedback, etc., which record the entire process from the birth to the maturity of the software. By deeply analyzing these documents, we can more comprehensively and meticulously understand the performance of the software in various quality indicators, such as functionality, reliability, ease of use, maintainability, etc.

Claims

1. A method for evaluating the quality of power grid dispatching management software based on hierarchical analysis method, characterized in that: The steps include: Step 1: Construct a multi-level quality analysis model based on the documentation of different stages in the software life cycle; Step 2: For the multi-level quality analysis model in step 1, experts score based on the evaluation indicators, construct a judgment matrix, and perform consistency test; Step 3: Extract the document summary based on the Randeng-Pegasus-523M-Chinese model, automatically extract keywords from the document based on the TextRank algorithm, and quickly evaluate the software-related documents; Step 4: Evaluate the software quality indicators; based on the results of step 3, evaluate the quality of the software process related documents one by one, and apply the judgment matrix based on step 2 to the document evaluation results to obtain the evaluation indicators of the comprehensive quality of the software.

2. The method for evaluating the quality of power grid dispatching management software based on the analytic hierarchy process according to claim 1, characterized in that: In step 1, the multi-level quality analysis model adopts the waterfall model, which divides the entire software cycle into six stages: software planning, requirements analysis, software design, program coding, software testing, and operation and maintenance; The software planning phase includes a feasibility study report and a project development plan; the requirements analysis phase includes business requirements documents, market requirements documents, user requirements documents, functional requirements specifications, non-functional requirements specifications, requirements specifications, use case and scenario documents, interface prototypes and designs, data dictionaries, and requirements tracking matrices; the software design phase includes system design documents, architecture design documents, component design documents, interface design documents, data design documents, use case implementation documents, design decision documents, and design specification documents; The software testing phase includes test plan documents, test case documents, test script documents, test matrix documents, test execution reports, defect management documents, test summary documents, and regression test documents; The operation and maintenance phase includes user manuals, maintenance plans, troubleshooting guides, system monitoring reports, change management records, security management documents, and business continuity plans.

3. The method for evaluating the quality of power grid dispatching management software based on the analytic hierarchy process according to claim 1, characterized in that: In step 2, experts score based on the evaluation indicators, and the judgment matrix is ​​constructed as follows: The meaning of the numerical scale of the pairwise comparison scores is listed in Table 1: Table 1 Meaning of digital scale Based on the expert scoring data, a judgment matrix is ​​generated:

4. The method for evaluating the quality of power grid dispatching management software based on the analytic hierarchy process according to claim 1, characterized in that: In step 2, the consistency check includes a hierarchical single-order consistency check and a hierarchical total-order consistency check.

5. The method for evaluating the quality of power grid dispatching management software based on the analytic hierarchy process as claimed in claim 4, characterized in that: The hierarchical single sorting consistency test is as follows: Calculate the consistency index CI: Find and calculate the average random consistency index RI, as listed in Table 2: Table 2 Average random consistency index RI standard value Finally, calculate the consistency ratio CR: When CR<0.1, the consistency of the judgment matrix is ​​acceptable. Otherwise, the judgment matrix is ​​appropriately adjusted and corrected. After the correction, the consistency test is performed again until the consistency test passes.

6. The method for evaluating the quality of power grid dispatching management software based on the analytic hierarchy process according to claim 4, characterized in that: The hierarchical total ranking consistency test is as follows: hierarchical total ranking is the process of determining the ranking weights of all factors in a certain layer for the overall goal. If the previous layer A contains n factors B1, B2, ..., B n , whose hierarchical weight values ​​are b1, b2, ..., b n , if some factors of the next level C are important for B # The consistency index of a single sort is CI # (i=1,2,3……n), the corresponding average random consistency index is RI # , then the total random consistency ratio of the C level is: When CR<0.1, it is considered that the hierarchical total ranking results have satisfactory consistency and the analysis results are acceptable.

7. The method for evaluating the quality of power grid dispatching management software based on the analytic hierarchy process according to claim 1, characterized in that: In step 3, the summary of the document is extracted based on the Randeng-Pegasus-523M-Chinese model as follows: collect large-scale Chinese text data, train through the GSG task in the pre-training stage, delete some sentences from the input text, let the model predict these deleted sentences, so that the model can learn how to generate sentences consistent with the context in the pre-training stage; build a software process document summary dataset to fine-tune the model, optimize the model parameters through supervised learning, and make it perform best in the summary generation task; evaluate the model using the ROUGE score based on the quality of the generated summary, and adjust the model parameters and training strategy based on the results.

8. The method for evaluating the quality of power grid dispatching management software based on the analytic hierarchy process according to claim 1, characterized in that: In step 3, automatically extracting keywords from documents based on the TextRank algorithm specifically includes the following steps: Step 31: perform data preprocessing operations such as word segmentation, part-of-speech tagging, and stop word removal on the text; Step 32: construct a candidate keyword graph G = (V, E), where V is a node set (consisting of candidate keywords), and use the co-occurrence relationship to construct an edge between any two nodes. An edge exists between two nodes only if their corresponding words co-occur in a window of length K, where K represents the window size, i.e., a maximum of K co-occurring words; Step 33, iteratively calculate the weight of each node until convergence; Step 34: Arrange the node weights in reverse order to obtain the top N words as text keywords.

9. The method for evaluating the quality of power grid dispatching management software based on the analytic hierarchy process according to claim 1, characterized in that: In step 4, based on the extracted summary and keyword information, the documents at different stages of the software process are quickly evaluated, and the evaluation indicators corresponding to each software development stage at the lowest level are counted; the evaluation value of the software quality at each stage of the third level is calculated using the indicator value of the fourth level and the weight matrix between the fourth level and the third level; then the second level evaluation value is calculated using the calculated third level evaluation value and the weight matrix between the second level and the third level; finally, the overall evaluation value of the software quality is calculated using the weight matrix between the first level and the second level; a consistent score range is established for each evaluation, and the scores of the evaluation indicators are taken as follows: Table 3 Evaluation index value standards 10. A system for implementing the power grid dispatching management software quality assessment method based on hierarchical analysis method as claimed in claim 1, characterized in that: include: Model building module, consistency checking module, document extraction module and software quality assessment module; The model building module builds a multi-level quality analysis model based on the document data at different stages of the software life cycle; the consistency verification module performs expert scoring based on the evaluation indicators, constructs a judgment matrix, and performs consistency verification; the document extraction module extracts the document summary based on the Randeng-Pegasus-523M-Chinese model, automatically extracts keywords in the document based on the TextRank algorithm, and quickly evaluates software-related documents; the software quality assessment module realizes the assessment of software quality.

Images