Method and device for evaluating software reliability guarantee model in power industry

Abstract

The invention provides an evaluation method and device for a software reliability guarantee model in the power industry, and the method comprises the steps: integrating key performance indexes of a plurality of artificial intelligence evaluation models, and determining a unified evaluation standard; a historical data set of the power equipment and the target task scene are adopted for training, a software reliability guarantee model is obtained, and the historical data set at least comprises historical operation parameters of software of the power equipment and historical performance parameters of the software; and based on the evaluation standard and by adopting the integrated evaluation model, repeatedly evaluating the performance of the software reliability guarantee model for multiple times to obtain a plurality of evaluation results, and calibrating the evaluation results by adopting a cross validation method to obtain a final evaluation result. According to the method, the performance evaluation consistency, the test efficiency and the overall system performance of the software reliability guarantee model can be effectively improved, and the software reliability of the power industry is improved.

Description

Technical Field

[0001] The present application relates to the field of software reliability in the electric power industry, and specifically, to an evaluation method for a software reliability assurance model in the electric power industry, an evaluation device for a software reliability assurance model in the electric power industry, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the rapid development of information technology and artificial intelligence technology, the power industry has gradually applied more and more intelligent means to improve the reliability and safety of the system. Especially in the aspect of software reliability assurance of power system, the introduction of artificial intelligence methods has greatly improved the efficiency and accuracy of tasks such as fault prediction and automatic repair.

[0003] The existing artificial intelligence methods have the following main problems in the evaluation of software reliability assurance models in the power industry:

[0004] Poor consistency in evaluation results: Due to the different evaluation standards and tasks of different benchmarks, the evaluation results of the same AI chip under different benchmarks may vary greatly, affecting the accurate evaluation of model performance.

[0005] Insufficient coverage of task types: Existing benchmarks mainly focus on general AI tasks and lack task types for specific application scenarios in the power industry, such as power equipment fault prediction and load management.

[0006] Lack of simulation models: Actual benchmark runs require a lot of time and financial resources, while existing benchmarks rarely consider simulation and performance bottleneck analysis of large-scale heterogeneous systems. Summary of the invention

[0007] The main purpose of this application is to provide a method for evaluating a software reliability assurance model for the electric power industry, an evaluation device for a software reliability assurance model for the electric power industry, a computer-readable storage medium and an electronic device, so as to at least solve the problems in the prior art of poor consistency of evaluation results, insufficient coverage of task types and lack of simulation models in the evaluation of software reliability assurance models for the electric power industry.

[0008] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for evaluating a software reliability assurance model in the power industry is provided, including: integrating key performance indicators of multiple artificial intelligence evaluation models to determine a unified evaluation standard; using historical data sets of power equipment and target task scenarios for training to obtain a software reliability assurance model, wherein the historical data sets at least include historical operating parameters of the software of the power equipment and historical performance parameters of the software; based on the evaluation standard and using an integrated evaluation model, repeatedly evaluating the performance of the software reliability assurance model to obtain multiple evaluation results, and calibrating the evaluation results using a cross-validation method to obtain a final evaluation result.

[0009] Optionally, historical data sets of power equipment and target task scenarios are used for training to obtain a software reliability assurance model, including: obtaining historical fault data and real-time monitoring data of the power equipment, and using the historical fault data and the real-time monitoring data to build an initial fault prediction model; obtaining historical fault scenarios and historical program repair data of the power equipment, and using the historical fault scenarios and the historical program repair data to build an initial automatic program repair model; obtaining historical load change data and historical load change scenarios of the power equipment, and using the historical load change data and the historical load change scenarios to build an initial load management model; determining the software reliability assurance model based on the initial fault prediction model, the initial automatic program repair model and the initial load management model.

[0010] Optionally, the software reliability assurance model is determined based on the initial fault prediction model, the initial automatic program repair model and the initial load management model, including: optimizing the initial fault prediction model, the initial automatic program recovery model and the initial load management model respectively to obtain a target fault prediction model, a target automatic program repair model and a target load management model; integrating the target fault prediction model, the target automatic program repair model and the target load management model to obtain the software reliability assurance model.

[0011] Optionally, after training with historical data sets of power equipment and target task scenarios to obtain a software reliability assurance model, the method further includes: constructing a virtual simulation environment based on a large-scale heterogeneous system; applying the software reliability assurance model in the virtual simulation environment for simulation to obtain simulation data, wherein the simulation data includes at least hardware operation data of the large-scale heterogeneous system, software operation data of the large-scale heterogeneous system, and operation efficiency of the software reliability assurance model.

[0012] Optionally, based on the evaluation standard and using an integrated evaluation model, the performance of the software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the evaluation results are calibrated by a cross-validation method to obtain a final evaluation result, including: based on the evaluation standard and using an integrated evaluation model, the performance of the software reliability assurance model is repeatedly evaluated multiple times, and a high-precision data collection tool is used to monitor and record in real time the various performance indicators of the software reliability assurance model during the evaluation process to obtain multiple evaluation results, and the evaluation results at least include the various performance indicators of the software reliability assurance model; the evaluation results are calibrated by a cross-validation method to obtain a final evaluation result; the final evaluation result is cleaned and analyzed by a data analysis tool to generate a performance analysis report; and the final evaluation result and the performance analysis report are visualized by a data visualization tool.

[0013] Optionally, after calibrating the evaluation results by a cross-validation method to obtain the final evaluation results, the method further includes: determining an optimization scheme for the large-scale heterogeneous system based on simulation data and the final evaluation results, the optimization scheme comprising at least hardware optimization data and software optimization data for the large-scale heterogeneous system; and optimizing the hardware and software of the large-scale heterogeneous system respectively by using the optimization scheme to obtain an optimized large-scale heterogeneous system, wherein the performance of the software reliability assurance model when running in the optimized large-scale heterogeneous system is greater than the performance of the software reliability assurance model when running in the large-scale heterogeneous system before optimization.

[0014] Optionally, the key performance indicators of multiple artificial intelligence evaluation models are integrated to determine a unified evaluation standard, including: obtaining the key performance indicators of each artificial intelligence evaluation model, the key performance indicators including at least one of the following: computing speed, efficiency, latency, throughput, model training and throughput speed; analyzing the key performance indicators of each artificial intelligence evaluation model to determine a set of indicators covering the key performance indicators of all artificial intelligence evaluation models; and determining a unified evaluation standard based on the indicator set.

[0015] According to another aspect of the present application, there is provided an evaluation device for a software reliability assurance model in the electric power industry, comprising: a first processing unit, for integrating key performance indicators of multiple artificial intelligence evaluation models to determine a unified evaluation standard; a second processing unit, for training with historical data sets of electric power equipment and target task scenarios to obtain a software reliability assurance model, wherein the historical data sets include at least historical operating parameters of the software of the electric power equipment and historical performance parameters of the software; a third processing unit, for repeatedly evaluating the performance of the software reliability assurance model based on the evaluation standard and using an integrated evaluation model to obtain multiple evaluation results, and calibrating the evaluation results using a cross-validation method to obtain a final evaluation result.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute any one of the evaluation methods for the software reliability assurance model of the electric power industry.

[0017] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include an evaluation method for executing any one of the above-mentioned software reliability assurance models for the electric power industry.

[0018] Applying the technical solution of the present application, the evaluation method of the software reliability assurance model in the above-mentioned power industry first integrates the key performance indicators of multiple artificial intelligence evaluation models, determines a unified evaluation standard, and integrates all artificial intelligence evaluation models to obtain an integrated evaluation model; uses the historical data set of power equipment and the target task scenario for training to obtain a software reliability assurance model, and the historical data set at least includes the historical operating parameters of the software of the power equipment and the historical performance parameters of the software; based on the evaluation standard and using the integrated evaluation model, the performance of the software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the evaluation results are calibrated by the cross-validation method to obtain the final evaluation result. This method can effectively improve the performance evaluation consistency, test efficiency and overall system performance of the software reliability assurance model by integrating different evaluation systems, covering a variety of software assurance tasks, and designing performance simulation models, significantly improving the overall evaluation efficiency and accuracy of the software reliability assurance model in the power industry, and solving the problems of poor consistency of evaluation results, insufficient coverage of task types, and lack of simulation models in the evaluation of software reliability assurance models in the power industry in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A hardware structure block diagram of a mobile terminal for executing an evaluation method for a software reliability assurance model in the electric power industry provided in an embodiment of the present application is shown;

[0021] Figure 2 A flow chart of a method for evaluating a software reliability assurance model in the electric power industry provided according to an embodiment of the present application is shown;

[0022] Figure 3 A structural block diagram of an evaluation device for a software reliability assurance model in the electric power industry provided according to an embodiment of the present application is shown.

[0023] The above drawings include the following reference numerals:

[0024] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0025] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] As introduced in the background technology, currently, the mainstream artificial intelligence performance evaluation methods mainly include the following:

[0029] 1.MLPerf: MLPerf is a widely used machine learning performance evaluation benchmark that covers multiple tasks from training to reasoning. MLPerf provides a unified performance evaluation method for different hardware and software platforms through standardized data sets, tasks, and metrics.

[0030] 2. AIPerf: AIPerf is another benchmark used to evaluate the performance of artificial intelligence systems, with a particular focus on the performance of different AI models in various application scenarios. AIPerf not only evaluates the training and inference speed of the model, but also focuses on its energy efficiency and latency in different hardware environments.

[0031] 3. HPL-AI: HPL-AI is an artificial intelligence performance evaluation tool based on high-performance computing (HPC). It evaluates the performance of AI models under high load by simulating a large-scale parallel computing environment, and is particularly suitable for application scenarios that require high computing power.

[0032] There are some key shortcomings and limitations in the current evaluation of software reliability assurance models in the power industry, as follows:

[0033] 1. Poor consistency of evaluation results:

[0034] There are differences in evaluation standards and tasks between existing benchmarks such as MLPerf, AIPerf, and HPL-AI. The evaluation results of the same AI chip under these benchmarks may vary greatly, affecting the accurate evaluation of model performance. This inconsistency makes it difficult to form a unified performance evaluation standard, thus limiting the fair comparison of different AI chips and models.

[0035] 2. Insufficient coverage of task types:

[0036] Current benchmarks mainly focus on general AI tasks, but lack task types for specific application scenarios in the power industry, such as power equipment failure prediction, load management, etc. This results in the lack of pertinence in existing evaluation methods in practical applications, and the inability to fully evaluate the actual performance and utility of the model in the power industry.

[0037] 3. Lack of simulation models:

[0038] Actual evaluation runs require a lot of time and money, and existing benchmarks rarely consider simulation and performance bottleneck analysis of large-scale heterogeneous systems. This makes it difficult to fully evaluate the system in a real environment and to accurately predict and resolve potential performance bottlenecks and failure points.

[0039] 4. High system complexity and computing resource requirements:

[0040] When dealing with large-scale and complex distribution networks, existing technologies require high computing resources, are complex, and difficult to maintain, which not only increases the development and operation costs of the system, but also places higher demands on the stability and reliability of the system.

[0041] 5. Real-time and response speed issues:

[0042] For online simulation systems, existing technologies may not respond quickly enough under high load and complex operations, affecting user experience. The inability to quickly process large amounts of data and generate accurate simulation results limits the practicality of the system.

[0043] The prior art still has obvious deficiencies in the evaluation of software reliability assurance models in the electric power industry, but in terms of consistency of evaluation results, coverage of task types, simulation model design, system complexity, computing resource requirements, and real-time performance. In order to solve the problems of poor consistency of evaluation results, insufficient coverage of task types, and lack of simulation models in the evaluation of software reliability assurance models in the electric power industry in the prior art, the embodiments of the present application provide an evaluation method for a software reliability assurance model in the electric power industry, an evaluation device for a software reliability assurance model in the electric power industry, a computer-readable storage medium, and an electronic device.

[0044] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0045] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of an evaluation method for a software reliability assurance model in the electric power industry according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0046] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the evaluation method of the software reliability assurance model of the power industry in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific example of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0047] In this embodiment, a method for evaluating a software reliability assurance model for the electric power industry running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] Figure 2 1 is a flow chart of a method for evaluating a software reliability assurance model for the electric power industry according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0049] Step S201, integrating the key performance indicators of multiple artificial intelligence evaluation models to determine a unified evaluation standard;

[0050] Specifically, multiple AI evaluation models include: MLPerf, AIPerf, HPL-AI and other different evaluation systems. There are differences between current benchmarks such as MLPerf, AIPerf and HPL-AI. The evaluation results of the same AI chip under these methods may be biased, affecting the accurate evaluation of the performance of the software assurance model. By integrating different evaluation systems such as MLPerf, AIPerf, HPL-AI, and establishing unified evaluation standards, we can ensure the consistency and accuracy of the evaluation results.

[0051] Specifically, in step S201, in order to integrate different evaluation systems such as MLPerf, AIPerf, and HPL-AI and establish a unified evaluation standard, the following steps are included:

[0052] 1. Collection of evaluation results and key performance indicators:

[0053] When integrating evaluation systems such as MLPerf, AIPerf, and HPL-AI, the following evaluation results and key performance indicators need to be collected:

[0054] Computing speed: such as the number of operations performed per second (FLOPs), the number of images processed per second, etc., used to measure the processing power of the model or chip.

[0055] Latency: The response time of a model or chip when performing a specific task, usually measured in milliseconds.

[0056] Throughput: The amount of data that a system can process per unit time, usually measured in MB / s or GB / s.

[0057] Energy efficiency: The energy consumed by a system per unit time, usually measured in Watt or Joule, is used to measure the energy efficiency of the system.

[0058] Model training speed: the training progress of the model within a certain period of time, such as the number of training samples processed per second.

[0059] Model inference speed: How fast the model performs inference in real-world applications, such as the number of predictions generated per second.

[0060] Memory usage: The memory resources required by a model or chip when running, usually measured in MB or GB.

[0061] Accuracy: The prediction accuracy of the model on a specific task, such as classification accuracy, regression error, etc.

[0062] 2. Steps of data fusion:

[0063] After collecting the key performance indicators of the above different evaluation systems, the following steps are used to integrate them:

[0064] (1) Standardization of indicators:

[0065] Since the measurement units and ranges of indicators in different evaluation systems may be different, these indicators need to be standardized first:

[0066] Unified units: Convert indicators in different evaluation systems into unified units. For example, latency is unified into milliseconds and throughput is unified into MB / s.

[0067] Normalization: Normalize the indicators and map them to a uniform scale range (such as [0,1]) to ensure comparability between different indicators. The commonly used method is Min-Max normalization:

[0068]

[0069] Among them, X is the original indicator value and X' is the normalized value.

[0070] (2) Weight distribution:

[0071] Determine the weight of each indicator through expert scoring or statistical analysis:

[0072] Expert scoring method: Invite experts in the field to score the importance of different indicators, and use the analytic hierarchy process (AHP) to determine the weight of each indicator. For example, in the power industry, energy efficiency and computing speed may be more important than other indicators.

[0073] Statistical analysis: Use historical data to conduct statistical analysis on the importance of each indicator, such as through correlation analysis, principal component analysis and other methods to determine the main indicators that affect the evaluation results.

[0074] (3) Indicator Fusion:

[0075] The standardized indicators are weighted averaged according to the determined weights to obtain a unified comprehensive indicator value. The specific fusion methods include:

[0076] Weighted average method: Take a weighted average of the indicators of each model and calculate the comprehensive score. The formula is as follows:

[0077]

[0078] Among them, w_i is the weight of the \i-th indicator, and X'_i is the normalized indicator value.

[0079] Multi-task learning (MTL): If machine learning methods are used, a multi-task learning model can be built, and the data of each evaluation system can be used as input. The model learns the relationship between different tasks and outputs a unified evaluation standard.

[0080] (4) Model calibration and verification:

[0081] The fused model needs to be calibrated and verified in practical applications:

[0082] Cross-validation: The accuracy and consistency of the fusion model are tested through the cross-validation method to ensure its applicability in different scenarios.

[0083] Simulation testing: Run the fusion model in a virtual simulation environment to simulate actual application scenarios in the power industry and verify its performance under different system configurations.

[0084] In addition to integrating MLPerf, AIPerf, and HPL-AI, other evaluation systems can also be introduced, such as SPEC CPU, TPC, etc., to formulate unified evaluation standards. By introducing more types of evaluation systems and integrating more key performance indicators, the evaluation indicator system can be further improved to make the evaluation results more representative and widely applicable. Increasing the types of evaluation systems can cover more evaluation scenarios and application requirements, and improve the uniformity and accuracy of evaluation standards.

[0085] Among them, integrating the key performance indicators of multiple artificial intelligence evaluation models and determining a unified evaluation standard include the following steps:

[0086] Step S2011, obtaining the key performance indicators of each of the above artificial intelligence evaluation models, wherein the above key performance indicators include at least one of the following: computing speed, efficiency, latency, throughput, model training and throughput speed;

[0087] Step S2012, analyzing the key performance indicators of each of the above artificial intelligence evaluation models, and determining a set of indicators covering the key performance indicators of all the above artificial intelligence evaluation models;

[0088] Step S2013, determining a unified evaluation standard according to the above indicator set.

[0089] Specifically, it includes integrating the key performance indicators of each evaluation system, selecting representative data sets and task scenarios, specifying standardized hardware configurations and software environments, and designing result calibration mechanisms to eliminate evaluation bias. By analyzing the key indicators of each evaluation system, a set of comprehensive and representative indicators is determined, and corresponding evaluation standards are formulated to ensure that consistent evaluation results can be obtained under different evaluation systems.

[0090] Step S202, using a historical data set of the power equipment and a target task scenario for training to obtain a software reliability assurance model, wherein the historical data set at least includes historical operating parameters of the software of the power equipment and historical performance parameters of the software;

[0091] Specifically, representative data sets relevant to the application scenarios of the power industry are selected as historical data sets, and standardized task scenarios are developed to ensure that the same data sets and tasks are used for testing under different evaluation systems to ensure the uniformity and fairness of the evaluation.

[0092] Current testing methods rarely consider the characteristics and requirements of different power application scenarios, and the types of tasks covered are insufficient. The above methods cover a variety of software assurance tasks related to the power industry, such as power equipment software fault prediction, automatic program repair, and power load management tasks, ensuring the high relevance of the evaluation tasks to the actual application scenarios. These tasks use historical fault data and real-time monitoring data to evaluate the performance of the model, simulate power load change scenarios, and evaluate the model's ability to predict and optimize power load.

[0093] When designing software assurance tasks related to the power industry, other types of tasks can be added, such as power equipment maintenance prediction, distributed energy management, smart grid optimization, etc. By expanding the task types, more application scenarios related to the power industry can be covered to ensure the diversity and high relevance of the evaluation tasks. Increasing the task types enables the evaluation method to adapt to more actual application scenarios and improve the comprehensiveness and practicality of the model performance evaluation.

[0094] The historical data set of power equipment and the target task scenario are used for training to obtain the software reliability assurance model, which includes the following steps:

[0095] Step S301, obtaining historical fault data and real-time monitoring data of the above-mentioned power equipment, and using the above-mentioned historical fault data and the above-mentioned real-time monitoring data to build an initial fault prediction model;

[0096] Among them, historical fault data and real-time monitoring data of power equipment are collected, a fault prediction model is built, and the prediction accuracy and efficiency of the model are evaluated in a test environment.

[0097] Step S302, obtaining historical fault scenarios and historical program repair data of the above-mentioned power equipment, and constructing an initial automatic program repair model using the above-mentioned historical fault scenarios and the above-mentioned historical program repair data;

[0098] Among them, common fault scenarios in the power system are simulated, the repair effect and time of the automatic program repair algorithm in these scenarios are tested, and its applicability and efficiency are evaluated.

[0099] Step S303, obtaining historical load change data and historical load change scenarios of the above-mentioned power equipment, and using the above-mentioned historical load change data and the above-mentioned historical load change scenarios to construct an initial load management model;

[0100] Among them, a simulation model of power load changes is constructed to test the performance of the load management algorithm in different load change scenarios, and to evaluate its prediction accuracy and optimization effect.

[0101] Step S304: determining the software reliability assurance model according to the initial fault prediction model, the initial automatic program repair model and the initial load management model.

[0102] Specifically, the software reliability assurance model consists of a fault prediction model, an automatic program repair model, and a load management model. It can evaluate the performance of the model in predicting software faults, evaluate the effectiveness and efficiency of the automatic program repair algorithm for common software faults in the power system, and evaluate the model's ability to predict and optimize power loads.

[0103] Determining the software reliability assurance model according to the initial fault prediction model, the initial automatic program repair model and the initial load management model includes the following steps:

[0104] Step S3041, respectively optimizing the initial fault prediction model, the initial automatic program recovery model, and the initial load management model to obtain a target fault prediction model, a target automatic program recovery model, and a target load management model;

[0105] Step S3042, integrating the target fault prediction model, the target automatic program repair model and the target load management model to obtain the software reliability assurance model.

[0106] Specifically, the software reliability assurance model consists of a fault prediction model, an automatic program repair model, and a load management model. It can predict software failures, perform automatic program repair for common software failures in the power system, and predict and optimize power loads.

[0107] Wherein, after the software reliability assurance model is obtained by training with the historical data set of the power equipment and the target task scenario, the above method further includes:

[0108] Step S401, constructing a virtual simulation environment based on a large-scale heterogeneous system;

[0109] Step S402, applying the above-mentioned software reliability assurance model in the above-mentioned virtual simulation environment to perform simulation to obtain simulation data, and the above-mentioned simulation data at least includes the hardware operation data of the above-mentioned large-scale heterogeneous system, the software operation data of the above-mentioned large-scale heterogeneous system and the operation efficiency of the above-mentioned software reliability assurance model.

[0110] Specifically, since the actual operation of the base test requires a lot of time and financial resources, it is necessary to design a performance simulation model for large-scale heterogeneous systems to check whether there are performance bottlenecks and potential failure points. Therefore, for large-scale heterogeneous systems, a performance simulation model is designed to simulate the actual operating environment, check potential performance bottlenecks and failure points, and optimize the evaluation process. Specifically, it includes heterogeneous system simulation, simulation environment construction, and bottleneck analysis and optimization.

[0111] Among them, large-scale heterogeneous systems are large-scale computer clusters. Before simulation, standardized hardware configuration and software environment need to be specified to ensure that performance evaluation is performed under the same conditions. Specifically, unified hardware and software configuration standards are formulated to ensure that all systems are tested in the same environment during the evaluation process, eliminating evaluation result deviations caused by hardware and software differences.

[0112] For large-scale heterogeneous systems, the design of performance simulation models specifically includes: 1. Heterogeneous system simulation: simulate heterogeneous computing environments such as CPU, GPU, FPGA, and evaluate the performance of the model under different hardware configurations. By building a variety of heterogeneous computing environments, test the operating performance of the model in these environments, and evaluate its adaptability and efficiency under different hardware configurations. 2. Simulation environment construction: build a virtual simulation environment to simulate the actual operation of the power system and detect the performance of the system under high load and complex scenarios. By building a virtual simulation platform similar to the actual power system operating environment, simulate high load and complex scenarios, and test the stability and performance bottlenecks of the system. 3. Bottleneck analysis and optimization: identify system performance bottlenecks through simulation testing and put forward corresponding optimization suggestions. Analyze simulation test data, identify performance bottlenecks, put forward optimization suggestions for hardware and software, and improve the overall performance of the system.

[0113] In addition to designing performance simulation models for large-scale heterogeneous systems, you can also use cloud computing platforms for simulation and use the elasticity and distributed computing capabilities of cloud resources for performance evaluation. By building a simulation environment on a cloud computing platform, you can simulate actual operation conditions, detect system performance bottlenecks and potential failure points, and optimize the evaluation process. By using the elasticity and distributed computing capabilities of cloud computing, you can perform large-scale simulations more efficiently, reduce the demand for local computing resources, and improve the flexibility and scalability of simulations.

[0114] Step S203, based on the above evaluation criteria and using the integrated evaluation model, repeatedly evaluate the performance of the above software reliability assurance model to obtain multiple evaluation results, and calibrate the above evaluation results using a cross-validation method to obtain a final evaluation result.

[0115] Specifically, in the prior art, the performance data collection and analysis during the evaluation process is not comprehensive enough, which affects the detailedness and accuracy of the evaluation report. This embodiment collects various performance data during the evaluation process, including computing speed, efficiency, delay, throughput, etc., conducts comprehensive analysis, provides a detailed evaluation report, and uses data analysis tools and visual displays to make the evaluation results more intuitive and convenient for analysis and decision-making.

[0116] Among them, based on the above evaluation standards and using the integrated evaluation model, the performance of the above software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the above evaluation results are calibrated by a cross-validation method to obtain the final evaluation results, including the following steps:

[0117] Step S501, based on the above evaluation criteria and using the integrated evaluation model, repeatedly evaluate the performance of the above software reliability assurance model for multiple times, and use a high-precision data collection tool to monitor and record in real time the various performance indicators of the above software reliability assurance model during the evaluation process, and obtain multiple evaluation results, wherein the above evaluation results at least include the above performance indicators of the above software reliability assurance model;

[0118] In the process of evaluation, the system performance data is collected in real time to ensure the integrity and accuracy of the data. High-precision data collection tools are used to monitor and record various performance indicators in real time during the evaluation process to ensure the integrity and accuracy of the data.

[0119] Step S502, calibrating the above evaluation results using a cross-validation method to obtain a final evaluation result;

[0120] Wherein, after the above evaluation results are calibrated by a cross-validation method to obtain the final evaluation results, the above method further includes the following steps:

[0121] Step S601, determining an optimization scheme for a large-scale heterogeneous system according to the simulation data and the final evaluation result, wherein the optimization scheme at least includes hardware optimization data and software optimization data for the large-scale heterogeneous system;

[0122] Step S602, using the above-mentioned optimization scheme to optimize the hardware and software of the above-mentioned large-scale heterogeneous system respectively, to obtain an optimized large-scale heterogeneous system, wherein the performance of the above-mentioned software reliability assurance model when running in the above-mentioned optimized large-scale heterogeneous system is greater than the performance of the above-mentioned software reliability assurance model when running in the above-mentioned large-scale heterogeneous system before optimization.

[0123] Specifically, the system optimization suggestions based on the evaluation results are not specific enough and it is difficult to effectively improve the performance of the software reliability assurance model. Based on the evaluation results, this embodiment proposes specific system optimization suggestions, including hardware configuration adjustment, algorithm optimization, etc., and ensures the optimization effect through multiple rounds of verification tests. Establish a continuous improvement mechanism, conduct performance evaluation and optimization regularly, and maintain efficient and stable operation of the system.

[0124] In the process of performance data collection and analysis, machine learning and big data analysis technologies can be introduced to automatically process and analyze large amounts of evaluation data. Through machine learning algorithms and big data analysis tools, the collected performance data can be intelligently processed and analyzed to generate more detailed and accurate performance analysis reports. Using machine learning and big data analysis technologies, large amounts of performance data can be processed and analyzed more efficiently, improving the depth and accuracy of data analysis.

[0125] Step S503, using a data analysis tool to clean and analyze the above final evaluation results to generate a performance analysis report;

[0126] Among them, data analysis tools are used to process the collected data and generate detailed performance analysis reports. Professional data analysis software is used to clean, process and analyze the collected data to generate detailed performance reports, providing a comprehensive view of system performance.

[0127] Step S504: Use a data visualization tool to visualize the final evaluation result and the performance analysis report.

[0128] Specifically, the evaluation results are displayed intuitively through charts and dashboards to facilitate analysis and decision-making. Using data visualization tools, the evaluation data is displayed in the form of charts, dashboards, etc., to help users quickly understand the evaluation results and make effective decisions.

[0129] Based on the evaluation results, specific suggestions for system optimization are put forward. First, according to the simulation data and evaluation data, a detailed system optimization plan is formulated: including hardware configuration adjustment, algorithm optimization, etc. Analyze the various performance data in the evaluation report, identify the weaknesses and bottlenecks in the system, put forward specific suggestions for hardware upgrades and algorithm optimization, and formulate a detailed optimization implementation plan. Then implement the optimization plan and conduct multiple rounds of verification tests to ensure the optimization effect. According to the optimization plan, the optimization measures are gradually implemented, and the optimization effect is evaluated through multiple rounds of verification tests to ensure the actual improvement of system performance. Finally, a continuous improvement mechanism is established: regular performance evaluation and optimization are carried out to maintain the efficient and stable operation of the system. Establish a regular performance evaluation and optimization process, regularly perform performance evaluation on the system, and continuously optimize the system according to the evaluation results to ensure its long-term efficient and stable operation.

[0130] In addition to making specific system optimization suggestions based on the evaluation results, adaptive optimization technology can also be introduced to enable the system to automatically adjust and optimize the configuration according to the real-time evaluation results. By introducing adaptive optimization algorithms, the system can dynamically adjust the hardware configuration and algorithm parameters according to real-time evaluation data to continuously optimize system performance. Adaptive optimization technology enables the system to automatically adjust the configuration according to actual performance during operation, maintaining efficient and stable operation of the system.

[0131] The above embodiments have been proven to be feasible and effective through a series of experiments, simulations and actual use. First, in a laboratory environment, we integrated multiple evaluation systems such as MLPerf, AIPerf and HPL-AI, and conducted multiple performance evaluations under standardized hardware configuration and software environment. The experimental results show that the unified evaluation standard effectively improves the consistency and accuracy of the evaluation results. Secondly, we designed and tested a variety of software assurance tasks related to the power industry, including power equipment software fault prediction, automatic program repair and power load management tasks. The test results in actual application scenarios show that these task designs are highly relevant and can accurately evaluate the performance of the model in actual applications. Third, by building a virtual simulation environment to simulate the operation of large-scale heterogeneous systems, we conducted multiple simulation tests to identify and optimize the performance bottlenecks of the system. The simulation results show that the system is stable under high load and complex scenarios, and the performance is significantly improved. In addition, in actual use, we collected and analyzed a large amount of performance data, and the generated detailed evaluation report provides an important reference for system optimization. Through these optimization measures, the overall performance and stability of the system have been significantly improved. In summary, this embodiment has shown good feasibility and effect in experiments, simulations and actual use, and has effectively solved key problems in the prior art.

[0132] The evaluation method of the software reliability assurance model of the above-mentioned power industry in the present application first integrates the key performance indicators of multiple artificial intelligence evaluation models, determines a unified evaluation standard, and integrates all artificial intelligence evaluation models to obtain an integrated evaluation model; uses the historical data set of power equipment and the target task scenario for training to obtain a software reliability assurance model, and the historical data set at least includes the historical operating parameters of the software of the power equipment and the historical performance parameters of the software; based on the evaluation standard and using the integrated evaluation model, the performance of the software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the evaluation results are calibrated by the cross-validation method to obtain the final evaluation result. This method can effectively improve the performance evaluation consistency, test efficiency and overall system performance of the software reliability assurance model by integrating different evaluation systems, covering a variety of software assurance tasks, and designing performance simulation models, significantly improving the overall evaluation efficiency and accuracy of the software reliability assurance model in the power industry, and solving the problems of poor consistency of evaluation results, insufficient coverage of task types, and lack of simulation models in the evaluation of software reliability assurance models in the power industry in the prior art.

[0133] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the evaluation method of the software reliability assurance model in the power industry of the present application will be described in detail below in combination with specific embodiments.

[0134] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0135] The embodiment of the present application also provides an evaluation device for a software reliability assurance model for the electric power industry. It should be noted that the evaluation device for the software reliability assurance model for the electric power industry in the embodiment of the present application can be used to execute the evaluation method for the software reliability assurance model for the electric power industry provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been described will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0136] The following introduces an evaluation device for a software reliability assurance model in the electric power industry provided in an embodiment of the present application.

[0137] Figure 3 Schematic diagram of an evaluation device for a software reliability assurance model in the electric power industry according to an embodiment of the present application. Figure 3 As shown, the device includes a first processing unit 10, a second processing unit 20 and a fourth processing unit 30, the first processing unit 10 is used to integrate the key performance indicators of multiple artificial intelligence evaluation models to determine a unified evaluation standard; the second processing unit 20 is used to use the historical data set of the power equipment and the target task scenario for training to obtain a software reliability assurance model, and the above historical data set at least includes the historical operating parameters of the software of the above power equipment and the historical performance parameters of the above software; the third processing unit 30 is used to repeatedly evaluate the performance of the above software reliability assurance model based on the above evaluation standard and using the integrated evaluation model to obtain multiple evaluation results, and calibrate the above evaluation results using a cross-validation method to obtain a final evaluation result.

[0138] The evaluation device of the software reliability assurance model of the power industry of the present application includes a first processing unit, a second processing unit and a fourth processing unit. The first processing unit is used to integrate the key performance indicators of multiple artificial intelligence evaluation models, determine a unified evaluation standard, and integrate all artificial intelligence evaluation models to obtain an integrated evaluation model; the second processing unit is used to use the historical data set of the power equipment and the target task scenario for training to obtain the software reliability assurance model, and the historical data set at least includes the historical operating parameters of the software of the power equipment and the historical performance parameters of the software; the third processing unit is used to repeatedly evaluate the performance of the software reliability assurance model based on the evaluation standard and the integrated evaluation model, obtain multiple evaluation results, and calibrate the evaluation results by cross-validation to obtain the final evaluation result. The device can effectively improve the performance evaluation consistency, test efficiency and overall system performance of the software reliability assurance model by integrating different evaluation systems, covering a variety of software assurance tasks, and designing performance simulation models, significantly improving the overall evaluation efficiency and accuracy of the software reliability assurance model of the power industry, and solving the problems of poor consistency of evaluation results, insufficient coverage of task types, and lack of simulation models in the evaluation of the software reliability assurance model of the power industry in the prior art.

[0139] In some examples, the second processing unit includes a first acquisition module, a second acquisition module, a third acquisition module and a first determination module, wherein the first acquisition module is used to acquire the historical fault data and real-time monitoring data of the above-mentioned power equipment, and use the above-mentioned historical fault data and the above-mentioned real-time monitoring data to build an initial fault prediction model; the second acquisition module is used to acquire the historical fault scenarios and historical program repair data of the above-mentioned power equipment, and use the above-mentioned historical fault scenarios and the above-mentioned historical program repair data to build an initial automatic program repair model; the third acquisition module is used to acquire the historical load change data and historical load change scenarios of the above-mentioned power equipment, and use the above-mentioned historical load change data and the above-mentioned historical load change scenarios to build an initial load management model; the first determination module is used to determine the above-mentioned software reliability assurance model according to the above-mentioned initial fault prediction model, the above-mentioned initial automatic program repair model and the above-mentioned initial load management model. The software reliability assurance model is composed of a fault prediction model, an automatic program repair model and a load management model, and can evaluate the performance of the model in predicting software faults, evaluate the effect and efficiency of the automatic program repair algorithm for common software faults in the power system, and evaluate the model's ability to predict and optimize power loads.

[0140] In some examples, the first determination module includes an optimization submodule and an integration submodule, wherein the optimization submodule is used to optimize the above-mentioned initial fault prediction model, the above-mentioned initial automatic program recovery model, and the above-mentioned initial load management model, respectively, to obtain a target fault prediction model, a target automatic program repair model, and a target load management model; and the integration submodule is used to integrate the above-mentioned target fault prediction model, the above-mentioned target automatic program repair model, and the above-mentioned target load management model, to obtain the above-mentioned software reliability assurance model. The software reliability assurance model is composed of a fault prediction model, an automatic program repair model, and a load management model, and can predict software faults, perform automatic program repair for common software faults in the power system, and predict and optimize power loads.

[0141] In some examples, the device further includes a construction module and a simulation module. The construction module is used to construct a virtual simulation environment based on a large-scale heterogeneous system after training with the historical data set of the power equipment and the target task scenario to obtain the software reliability assurance model; the simulation module is used to apply the software reliability assurance model in the virtual simulation environment for simulation to obtain simulation data, and the simulation data at least includes the hardware operation data of the large-scale heterogeneous system, the software operation data of the large-scale heterogeneous system, and the operation efficiency of the software reliability assurance model. For large-scale heterogeneous systems, a performance simulation model is designed to simulate the actual operation environment, check potential performance bottlenecks and failure points, and optimize the evaluation process.

[0142] In this embodiment, the third processing unit includes an evaluation module, a calibration module, a first processing module and a visualization module. The evaluation module is used to repeatedly evaluate the performance of the software reliability assurance model based on the above evaluation criteria and using an integrated evaluation model, and use a high-precision data collection tool to monitor and record the various performance indicators of the software reliability assurance model in the evaluation process in real time to obtain multiple evaluation results, and the above evaluation results at least include the above performance indicators of the software reliability assurance model; the calibration module is used to calibrate the above evaluation results using a cross-validation method to obtain the final evaluation results; the first processing module is used to use a data analysis tool to clean and analyze the above final evaluation results and generate a performance analysis report; the visualization module is used to use a data visualization tool to visualize the above final evaluation results and the above performance analysis report. Using data visualization tools, the evaluation data is displayed in the form of charts, dashboards, etc., to help users quickly understand the evaluation results and make effective decisions.

[0143] An optional solution, the device further includes a second determination module and an optimization module, the second determination module is used to calibrate the evaluation results by a cross-validation method, and after obtaining the final evaluation results, determine the optimization scheme of the large-scale heterogeneous system according to the simulation data and the final evaluation results, wherein the optimization scheme at least includes the hardware optimization data and software optimization data of the large-scale heterogeneous system; the optimization module is used to optimize the hardware and software of the large-scale heterogeneous system respectively by using the optimization scheme to obtain the optimized large-scale heterogeneous system, wherein the performance of the software reliability assurance model when running in the optimized large-scale heterogeneous system is greater than the performance of the software reliability assurance model when running in the large-scale heterogeneous system before optimization. Based on the evaluation results, specific system optimization suggestions are put forward, including hardware configuration adjustment, algorithm optimization, etc., and the optimization effect is ensured through multiple rounds of verification tests. A continuous improvement mechanism is established to conduct performance evaluation and optimization regularly to maintain efficient and stable operation of the system.

[0144] As an optional solution, the first processing unit includes a fourth acquisition module, a third determination module and a fourth determination module, the fourth acquisition module is used to obtain the key performance indicators of each of the above-mentioned artificial intelligence evaluation models, and the above-mentioned key performance indicators include at least one of the following: computing speed, efficiency, delay, throughput, model training and throughput speed; the third determination module is used to analyze the key performance indicators of each of the above-mentioned artificial intelligence evaluation models, and determine a set of indicators covering the key performance indicators of all the above-mentioned artificial intelligence evaluation models; the fourth determination module is used to determine a unified evaluation standard based on the above-mentioned indicator set. By analyzing the key indicators of each evaluation system, a set of comprehensive and representative indicators is determined, and corresponding evaluation standards are formulated to ensure that consistent evaluation results can be obtained under different evaluation systems.

[0145] The evaluation device of the software reliability assurance model of the power industry includes a processor and a memory. The first processing unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0146] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problems of poor consistency of evaluation results, insufficient task type coverage, and lack of simulation models in the software reliability assurance model evaluation of the power industry in the prior art.

[0147] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0148] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the evaluation method of the software reliability assurance model of the power industry.

[0149] Specifically, the evaluation methods of the software reliability assurance model in the power industry include:

[0150] Step S201, integrating the key performance indicators of multiple artificial intelligence evaluation models to determine a unified evaluation standard;

[0151] Specifically, multiple AI evaluation models include: MLPerf, AIPerf, HPL-AI and other different evaluation systems. There are differences between current benchmarks such as MLPerf, AIPerf and HPL-AI. The evaluation results of the same AI chip under these methods may be biased, affecting the accurate evaluation of the performance of the software assurance model. By integrating different evaluation systems such as MLPerf, AIPerf, HPL-AI, and establishing unified evaluation standards, we can ensure the consistency and accuracy of the evaluation results.

[0152] Step S202, using a historical data set of the power equipment and a target task scenario for training to obtain a software reliability assurance model, wherein the historical data set at least includes historical operating parameters of the software of the power equipment and historical performance parameters of the software;

[0153] Specifically, representative data sets relevant to the application scenarios of the power industry are selected as historical data sets, and standardized task scenarios are developed to ensure that the same data sets and tasks are used for testing under different evaluation systems to ensure the uniformity and fairness of the evaluation.

[0154] Step S203, based on the above evaluation criteria and using the integrated evaluation model, repeatedly evaluate the performance of the above software reliability assurance model to obtain multiple evaluation results, and calibrate the above evaluation results using a cross-validation method to obtain a final evaluation result.

[0155] Specifically, in the prior art, the performance data collection and analysis during the evaluation process is not comprehensive enough, which affects the detailedness and accuracy of the evaluation report. This embodiment collects various performance data during the evaluation process, including computing speed, efficiency, delay, throughput, etc., conducts comprehensive analysis, provides a detailed evaluation report, and uses data analysis tools and visual displays to make the evaluation results more intuitive and convenient for analysis and decision-making.

[0156] Optionally, historical data sets of the power equipment and target task scenarios are used for training to obtain a software reliability assurance model, including: obtaining historical fault data and real-time monitoring data of the above-mentioned power equipment, and using the above-mentioned historical fault data and the above-mentioned real-time monitoring data to construct an initial fault prediction model; obtaining historical fault scenarios and historical program repair data of the above-mentioned power equipment, and using the above-mentioned historical fault scenarios and the above-mentioned historical program repair data to construct an initial automatic program repair model; obtaining historical load change data and historical load change scenarios of the above-mentioned power equipment, and using the above-mentioned historical load change data and the above-mentioned historical load change scenarios to construct an initial load management model; determining the above-mentioned software reliability assurance model based on the above-mentioned initial fault prediction model, the above-mentioned initial automatic program repair model and the above-mentioned initial load management model.

[0157] Optionally, the software reliability assurance model is determined based on the initial fault prediction model, the initial automatic program repair model and the initial load management model, including: optimizing the initial fault prediction model, the initial automatic program recovery model and the initial load management model respectively to obtain a target fault prediction model, a target automatic program repair model and a target load management model; integrating the target fault prediction model, the target automatic program repair model and the target load management model to obtain the software reliability assurance model.

[0158] Optionally, after training with historical data sets of power equipment and target task scenarios to obtain a software reliability assurance model, the method further includes: constructing a virtual simulation environment based on a large-scale heterogeneous system; applying the software reliability assurance model in the virtual simulation environment for simulation to obtain simulation data, wherein the simulation data at least includes hardware operation data of the large-scale heterogeneous system, software operation data of the large-scale heterogeneous system, and operation efficiency of the software reliability assurance model.

[0159] Optionally, based on the above-mentioned evaluation standards and using an integrated evaluation model, the performance of the above-mentioned software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the above-mentioned evaluation results are calibrated by a cross-validation method to obtain a final evaluation result, including: based on the above-mentioned evaluation standards and using an integrated evaluation model, the performance of the above-mentioned software reliability assurance model is repeatedly evaluated multiple times, and a high-precision data collection tool is used to monitor and record the various performance indicators of the above-mentioned software reliability assurance model during the evaluation process in real time to obtain multiple evaluation results, and the above-mentioned evaluation results at least include the above-mentioned performance indicators of the above-mentioned software reliability assurance model; the above-mentioned evaluation results are calibrated by a cross-validation method to obtain a final evaluation result; the above-mentioned final evaluation result is cleaned and analyzed by a data analysis tool to generate a performance analysis report; and the above-mentioned final evaluation result and the above-mentioned performance analysis report are visualized by a data visualization tool.

[0160] Optionally, after calibrating the above-mentioned evaluation results by a cross-validation method to obtain the final evaluation results, the above-mentioned method also includes: determining an optimization scheme for the large-scale heterogeneous system based on simulation data and the above-mentioned final evaluation results, the above-mentioned optimization scheme at least including hardware optimization data and software optimization data of the above-mentioned large-scale heterogeneous system; using the above-mentioned optimization scheme to optimize the hardware and software of the above-mentioned large-scale heterogeneous system respectively to obtain an optimized large-scale heterogeneous system, wherein the performance of the above-mentioned software reliability assurance model when running in the above-mentioned optimized large-scale heterogeneous system is greater than the performance of the above-mentioned software reliability assurance model when running in the above-mentioned large-scale heterogeneous system before optimization.

[0161] Optionally, the key performance indicators of multiple artificial intelligence evaluation models are integrated to determine a unified evaluation standard, including: obtaining the key performance indicators of each of the above-mentioned artificial intelligence evaluation models, the above-mentioned key performance indicators include at least one of the following: computing speed, efficiency, latency, throughput, model training and throughput speed; analyzing the key performance indicators of each of the above-mentioned artificial intelligence evaluation models to determine a set of indicators covering the key performance indicators of all the above-mentioned artificial intelligence evaluation models; and determining a unified evaluation standard based on the above-mentioned indicator set.

[0162] An embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes the evaluation method of the software reliability assurance model of the electric power industry when running.

[0163] Specifically, the evaluation methods of the software reliability assurance model in the power industry include:

[0164] Step S201, integrating the key performance indicators of multiple artificial intelligence evaluation models to determine a unified evaluation standard;

[0165] Specifically, multiple AI evaluation models include: MLPerf, AIPerf, HPL-AI and other different evaluation systems. There are differences between current benchmarks such as MLPerf, AIPerf and HPL-AI. The evaluation results of the same AI chip under these methods may be biased, affecting the accurate evaluation of the performance of the software assurance model. By integrating different evaluation systems such as MLPerf, AIPerf, HPL-AI, and establishing unified evaluation standards, we can ensure the consistency and accuracy of the evaluation results.

[0166] Step S202, using a historical data set of the power equipment and a target task scenario for training to obtain a software reliability assurance model, wherein the historical data set at least includes historical operating parameters of the software of the power equipment and historical performance parameters of the software;

[0167] Specifically, representative data sets relevant to the application scenarios of the power industry are selected as historical data sets, and standardized task scenarios are developed to ensure that the same data sets and tasks are used for testing under different evaluation systems to ensure the uniformity and fairness of the evaluation.

[0168] Step S203, based on the above evaluation criteria and using the integrated evaluation model, repeatedly evaluate the performance of the above software reliability assurance model to obtain multiple evaluation results, and calibrate the above evaluation results using a cross-validation method to obtain a final evaluation result.

[0169] Specifically, in the prior art, the performance data collection and analysis during the evaluation process is not comprehensive enough, which affects the detailedness and accuracy of the evaluation report. This embodiment collects various performance data during the evaluation process, including computing speed, efficiency, delay, throughput, etc., conducts comprehensive analysis, provides a detailed evaluation report, and uses data analysis tools and visual displays to make the evaluation results more intuitive and convenient for analysis and decision-making.

[0170] Optionally, historical data sets of the power equipment and target task scenarios are used for training to obtain a software reliability assurance model, including: obtaining historical fault data and real-time monitoring data of the above-mentioned power equipment, and using the above-mentioned historical fault data and the above-mentioned real-time monitoring data to construct an initial fault prediction model; obtaining historical fault scenarios and historical program repair data of the above-mentioned power equipment, and using the above-mentioned historical fault scenarios and the above-mentioned historical program repair data to construct an initial automatic program repair model; obtaining historical load change data and historical load change scenarios of the above-mentioned power equipment, and using the above-mentioned historical load change data and the above-mentioned historical load change scenarios to construct an initial load management model; determining the above-mentioned software reliability assurance model based on the above-mentioned initial fault prediction model, the above-mentioned initial automatic program repair model and the above-mentioned initial load management model.

[0171] Optionally, the software reliability assurance model is determined based on the initial fault prediction model, the initial automatic program repair model and the initial load management model, including: optimizing the initial fault prediction model, the initial automatic program recovery model and the initial load management model respectively to obtain a target fault prediction model, a target automatic program repair model and a target load management model; integrating the target fault prediction model, the target automatic program repair model and the target load management model to obtain the software reliability assurance model.

[0172] Optionally, after training with historical data sets of power equipment and target task scenarios to obtain a software reliability assurance model, the method further includes: constructing a virtual simulation environment based on a large-scale heterogeneous system; applying the software reliability assurance model in the virtual simulation environment for simulation to obtain simulation data, wherein the simulation data at least includes hardware operation data of the large-scale heterogeneous system, software operation data of the large-scale heterogeneous system, and operation efficiency of the software reliability assurance model.

[0173] Optionally, based on the above-mentioned evaluation standards and using an integrated evaluation model, the performance of the above-mentioned software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the above-mentioned evaluation results are calibrated by a cross-validation method to obtain a final evaluation result, including: based on the above-mentioned evaluation standards and using an integrated evaluation model, the performance of the above-mentioned software reliability assurance model is repeatedly evaluated multiple times, and a high-precision data collection tool is used to monitor and record the various performance indicators of the above-mentioned software reliability assurance model during the evaluation process in real time to obtain multiple evaluation results, and the above-mentioned evaluation results at least include the above-mentioned performance indicators of the above-mentioned software reliability assurance model; the above-mentioned evaluation results are calibrated by a cross-validation method to obtain a final evaluation result; the above-mentioned final evaluation result is cleaned and analyzed by a data analysis tool to generate a performance analysis report; and the above-mentioned final evaluation result and the above-mentioned performance analysis report are visualized by a data visualization tool.

[0174] Optionally, after calibrating the above-mentioned evaluation results by a cross-validation method to obtain the final evaluation results, the above-mentioned method also includes: determining an optimization scheme for the large-scale heterogeneous system based on simulation data and the above-mentioned final evaluation results, the above-mentioned optimization scheme at least including hardware optimization data and software optimization data of the above-mentioned large-scale heterogeneous system; using the above-mentioned optimization scheme to optimize the hardware and software of the above-mentioned large-scale heterogeneous system respectively to obtain an optimized large-scale heterogeneous system, wherein the performance of the above-mentioned software reliability assurance model when running in the above-mentioned optimized large-scale heterogeneous system is greater than the performance of the above-mentioned software reliability assurance model when running in the above-mentioned large-scale heterogeneous system before optimization.

[0175] Optionally, the key performance indicators of multiple artificial intelligence evaluation models are integrated to determine a unified evaluation standard, including: obtaining the key performance indicators of each of the above-mentioned artificial intelligence evaluation models, the above-mentioned key performance indicators include at least one of the following: computing speed, efficiency, latency, throughput, model training and throughput speed; analyzing the key performance indicators of each of the above-mentioned artificial intelligence evaluation models to determine a set of indicators covering the key performance indicators of all the above-mentioned artificial intelligence evaluation models; and determining a unified evaluation standard based on the above-mentioned indicator set.

[0176] An embodiment of the present invention provides a device, the device including a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0177] Step S201, integrating the key performance indicators of multiple artificial intelligence evaluation models to determine a unified evaluation standard;

[0178] Step S202, using a historical data set of the power equipment and a target task scenario for training to obtain a software reliability assurance model, wherein the historical data set at least includes historical operating parameters of the software of the power equipment and historical performance parameters of the software;

[0179] Step S203, based on the above evaluation criteria and using the integrated evaluation model, repeatedly evaluate the performance of the above software reliability assurance model to obtain multiple evaluation results, and calibrate the above evaluation results using a cross-validation method to obtain a final evaluation result.

[0180] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0181] Optionally, historical data sets of the power equipment and target task scenarios are used for training to obtain a software reliability assurance model, including: obtaining historical fault data and real-time monitoring data of the above-mentioned power equipment, and using the above-mentioned historical fault data and the above-mentioned real-time monitoring data to construct an initial fault prediction model; obtaining historical fault scenarios and historical program repair data of the above-mentioned power equipment, and using the above-mentioned historical fault scenarios and the above-mentioned historical program repair data to construct an initial automatic program repair model; obtaining historical load change data and historical load change scenarios of the above-mentioned power equipment, and using the above-mentioned historical load change data and the above-mentioned historical load change scenarios to construct an initial load management model; determining the above-mentioned software reliability assurance model based on the above-mentioned initial fault prediction model, the above-mentioned initial automatic program repair model and the above-mentioned initial load management model.

[0182] Optionally, the software reliability assurance model is determined based on the initial fault prediction model, the initial automatic program repair model and the initial load management model, including: optimizing the initial fault prediction model, the initial automatic program recovery model and the initial load management model respectively to obtain a target fault prediction model, a target automatic program repair model and a target load management model; integrating the target fault prediction model, the target automatic program repair model and the target load management model to obtain the software reliability assurance model.

[0183] Optionally, after training with historical data sets of power equipment and target task scenarios to obtain a software reliability assurance model, the method further includes: constructing a virtual simulation environment based on a large-scale heterogeneous system; applying the software reliability assurance model in the virtual simulation environment for simulation to obtain simulation data, wherein the simulation data at least includes hardware operation data of the large-scale heterogeneous system, software operation data of the large-scale heterogeneous system, and operation efficiency of the software reliability assurance model.

[0184] Optionally, based on the above-mentioned evaluation standards and using an integrated evaluation model, the performance of the above-mentioned software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the above-mentioned evaluation results are calibrated by a cross-validation method to obtain a final evaluation result, including: based on the above-mentioned evaluation standards and using an integrated evaluation model, the performance of the above-mentioned software reliability assurance model is repeatedly evaluated multiple times, and a high-precision data collection tool is used to monitor and record the various performance indicators of the above-mentioned software reliability assurance model during the evaluation process in real time to obtain multiple evaluation results, and the above-mentioned evaluation results at least include the above-mentioned performance indicators of the above-mentioned software reliability assurance model; the above-mentioned evaluation results are calibrated by a cross-validation method to obtain a final evaluation result; the above-mentioned final evaluation result is cleaned and analyzed by a data analysis tool to generate a performance analysis report; and the above-mentioned final evaluation result and the above-mentioned performance analysis report are visualized by a data visualization tool.

[0185] Optionally, after calibrating the above-mentioned evaluation results by a cross-validation method to obtain the final evaluation results, the above-mentioned method also includes: determining an optimization scheme for the large-scale heterogeneous system based on simulation data and the above-mentioned final evaluation results, the above-mentioned optimization scheme at least including hardware optimization data and software optimization data of the above-mentioned large-scale heterogeneous system; using the above-mentioned optimization scheme to optimize the hardware and software of the above-mentioned large-scale heterogeneous system respectively to obtain an optimized large-scale heterogeneous system, wherein the performance of the above-mentioned software reliability assurance model when running in the above-mentioned optimized large-scale heterogeneous system is greater than the performance of the above-mentioned software reliability assurance model when running in the above-mentioned large-scale heterogeneous system before optimization.

[0186] Optionally, the key performance indicators of multiple artificial intelligence evaluation models are integrated to determine a unified evaluation standard, including: obtaining the key performance indicators of each of the above-mentioned artificial intelligence evaluation models, the above-mentioned key performance indicators include at least one of the following: computing speed, efficiency, latency, throughput, model training and throughput speed; analyzing the key performance indicators of each of the above-mentioned artificial intelligence evaluation models to determine a set of indicators covering the key performance indicators of all the above-mentioned artificial intelligence evaluation models; and determining a unified evaluation standard based on the above-mentioned indicator set.

[0187] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0188] Step S201, integrating the key performance indicators of multiple artificial intelligence evaluation models to determine a unified evaluation standard;

[0189] Step S202, using a historical data set of the power equipment and a target task scenario for training to obtain a software reliability assurance model, wherein the historical data set at least includes historical operating parameters of the software of the power equipment and historical performance parameters of the software;

[0190] Step S203, based on the above evaluation criteria and using the integrated evaluation model, repeatedly evaluate the performance of the above software reliability assurance model to obtain multiple evaluation results, and calibrate the above evaluation results using a cross-validation method to obtain a final evaluation result.

[0191] Optionally, historical data sets of the power equipment and target task scenarios are used for training to obtain a software reliability assurance model, including: obtaining historical fault data and real-time monitoring data of the above-mentioned power equipment, and using the above-mentioned historical fault data and the above-mentioned real-time monitoring data to construct an initial fault prediction model; obtaining historical fault scenarios and historical program repair data of the above-mentioned power equipment, and using the above-mentioned historical fault scenarios and the above-mentioned historical program repair data to construct an initial automatic program repair model; obtaining historical load change data and historical load change scenarios of the above-mentioned power equipment, and using the above-mentioned historical load change data and the above-mentioned historical load change scenarios to construct an initial load management model; determining the above-mentioned software reliability assurance model based on the above-mentioned initial fault prediction model, the above-mentioned initial automatic program repair model and the above-mentioned initial load management model.

[0192] Optionally, the software reliability assurance model is determined based on the initial fault prediction model, the initial automatic program repair model and the initial load management model, including: optimizing the initial fault prediction model, the initial automatic program recovery model and the initial load management model respectively to obtain a target fault prediction model, a target automatic program repair model and a target load management model; integrating the target fault prediction model, the target automatic program repair model and the target load management model to obtain the software reliability assurance model.

[0193] Optionally, after training with historical data sets of power equipment and target task scenarios to obtain a software reliability assurance model, the method further includes: constructing a virtual simulation environment based on a large-scale heterogeneous system; applying the software reliability assurance model in the virtual simulation environment for simulation to obtain simulation data, wherein the simulation data at least includes hardware operation data of the large-scale heterogeneous system, software operation data of the large-scale heterogeneous system, and operation efficiency of the software reliability assurance model.

[0194] Optionally, based on the above-mentioned evaluation standards and using an integrated evaluation model, the performance of the above-mentioned software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the above-mentioned evaluation results are calibrated by a cross-validation method to obtain a final evaluation result, including: based on the above-mentioned evaluation standards and using an integrated evaluation model, the performance of the above-mentioned software reliability assurance model is repeatedly evaluated multiple times, and a high-precision data collection tool is used to monitor and record the various performance indicators of the above-mentioned software reliability assurance model during the evaluation process in real time to obtain multiple evaluation results, and the above-mentioned evaluation results at least include the above-mentioned performance indicators of the above-mentioned software reliability assurance model; the above-mentioned evaluation results are calibrated by a cross-validation method to obtain a final evaluation result; the above-mentioned final evaluation result is cleaned and analyzed by a data analysis tool to generate a performance analysis report; and the above-mentioned final evaluation result and the above-mentioned performance analysis report are visualized by a data visualization tool.

[0195] Optionally, after calibrating the above-mentioned evaluation results by a cross-validation method to obtain the final evaluation results, the above-mentioned method also includes: determining an optimization scheme for the large-scale heterogeneous system based on simulation data and the above-mentioned final evaluation results, the above-mentioned optimization scheme at least including hardware optimization data and software optimization data of the above-mentioned large-scale heterogeneous system; using the above-mentioned optimization scheme to optimize the hardware and software of the above-mentioned large-scale heterogeneous system respectively to obtain an optimized large-scale heterogeneous system, wherein the performance of the above-mentioned software reliability assurance model when running in the above-mentioned optimized large-scale heterogeneous system is greater than the performance of the above-mentioned software reliability assurance model when running in the above-mentioned large-scale heterogeneous system before optimization.

[0196] Optionally, the key performance indicators of multiple artificial intelligence evaluation models are integrated to determine a unified evaluation standard, including: obtaining the key performance indicators of each of the above-mentioned artificial intelligence evaluation models, the above-mentioned key performance indicators include at least one of the following: computing speed, efficiency, latency, throughput, model training and throughput speed; analyzing the key performance indicators of each of the above-mentioned artificial intelligence evaluation models to determine a set of indicators covering the key performance indicators of all the above-mentioned artificial intelligence evaluation models; and determining a unified evaluation standard based on the above-mentioned indicator set.

[0197] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0198] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0199] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0200] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0201] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0202] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0203] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0204] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0205] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0206] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0207] 1) The evaluation method of the software reliability assurance model of the power industry in the present application first integrates the key performance indicators of multiple artificial intelligence evaluation models, determines a unified evaluation standard, and integrates all artificial intelligence evaluation models to obtain an integrated evaluation model; uses the historical data set of power equipment and the target task scenario for training to obtain a software reliability assurance model, and the historical data set at least includes the historical operating parameters of the software of the power equipment and the historical performance parameters of the software; based on the evaluation standard and using the integrated evaluation model, the performance of the software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the evaluation results are calibrated by the cross-validation method to obtain the final evaluation results. This method can effectively improve the performance evaluation consistency, test efficiency and overall system performance of the software reliability assurance model by integrating different evaluation systems, covering a variety of software assurance tasks, and designing performance simulation models, significantly improving the overall evaluation efficiency and accuracy of the software reliability assurance model in the power industry, and solving the problems of poor consistency of evaluation results, insufficient coverage of task types, and lack of simulation models in the evaluation of software reliability assurance models in the power industry in the prior art.

[0208] 2) The evaluation device of the software reliability assurance model of the power industry of the present application includes a first processing unit, a second processing unit and a fourth processing unit. The first processing unit is used to integrate the key performance indicators of multiple artificial intelligence evaluation models, determine a unified evaluation standard, and integrate all artificial intelligence evaluation models to obtain an integrated evaluation model; the second processing unit is used to use the historical data set of the power equipment and the target task scenario for training to obtain the software reliability assurance model, and the historical data set at least includes the historical operating parameters of the software of the power equipment and the historical performance parameters of the software; the third processing unit is used to repeatedly evaluate the performance of the software reliability assurance model based on the evaluation standard and the integrated evaluation model to obtain multiple evaluation results, and the evaluation results are calibrated by the cross-validation method to obtain the final evaluation result. The device can effectively improve the performance evaluation consistency, test efficiency and overall system performance of the software reliability assurance model by integrating different evaluation systems, covering a variety of software assurance tasks, and designing performance simulation models, and significantly improves the overall evaluation efficiency and accuracy of the software reliability assurance model of the power industry, and solves the problems of poor consistency of evaluation results, insufficient coverage of task types, and lack of simulation models in the evaluation of the software reliability assurance model of the power industry in the prior art.

[0209] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for evaluating a software reliability assurance model in the electric power industry, characterized in that: include: Integrate the key performance indicators of multiple AI evaluation models and determine a unified evaluation standard; A software reliability assurance model is obtained by training using a historical data set of the power equipment and a target task scenario, wherein the historical data set includes at least historical operating parameters of the software of the power equipment and historical performance parameters of the software; Based on the evaluation standard and using an integrated evaluation model, the performance of the software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the evaluation results are calibrated using a cross-validation method to obtain a final evaluation result.

2. The method according to claim 1, characterized in that: The historical data set of power equipment and the target mission scenario are used for training to obtain the software reliability assurance model, including: Acquire historical fault data and real-time monitoring data of the power equipment, and use the historical fault data and the real-time monitoring data to build an initial fault prediction model; Acquire historical fault scenarios and historical program repair data of the power equipment, and construct an initial automatic program repair model using the historical fault scenarios and the historical program repair data; Acquire historical load change data and historical load change scenarios of the power equipment, and use the historical load change data and the historical load change scenarios to build an initial load management model; The software reliability assurance model is determined according to the initial fault prediction model, the initial automatic program repair model and the initial load management model.

3. The method according to claim 2, characterized in that Determining the software reliability assurance model according to the initial fault prediction model, the initial automatic program repair model, and the initial load management model includes: Respectively optimizing the initial fault prediction model, the initial automatic program recovery model, and the initial load management model to obtain a target fault prediction model, a target automatic program recovery model, and a target load management model; The target fault prediction model, the target automatic program repair model and the target load management model are integrated to obtain the software reliability assurance model.

4. The method according to claim 1, characterized in that: After the software reliability assurance model is obtained by training using the historical data set of the power equipment and the target task scenario, the method further includes: Build a virtual simulation environment based on large-scale heterogeneous systems; The software reliability assurance model is applied in the virtual simulation environment to perform simulation to obtain simulation data, which at least includes the hardware operation data of the large-scale heterogeneous system, the software operation data of the large-scale heterogeneous system and the operation efficiency of the software reliability assurance model.

5. The method according to claim 1, characterized in that Based on the evaluation criteria and using the integrated evaluation model, the performance of the software reliability assurance model is repeatedly evaluated multiple times to obtain multiple evaluation results, and the evaluation results are calibrated using a cross-validation method to obtain the final evaluation results, including: Based on the evaluation criteria and using an integrated evaluation model, the performance of the software reliability assurance model is repeatedly evaluated multiple times, and a high-precision data collection tool is used to monitor and record in real time the various performance indicators of the software reliability assurance model during the evaluation process to obtain multiple evaluation results, wherein the evaluation results at least include the various performance indicators of the software reliability assurance model; The evaluation results are calibrated using a cross-validation method to obtain a final evaluation result; Using data analysis tools to clean and analyze the final evaluation results and generate a performance analysis report; The final evaluation results and the performance analysis report are visualized using data visualization tools.

6. The method according to claim 5, characterized in that After the evaluation results are calibrated by a cross-validation method to obtain a final evaluation result, the method further includes: Determine an optimization scheme for the large-scale heterogeneous system according to the simulation data and the final evaluation result, wherein the optimization scheme at least includes hardware optimization data and software optimization data for the large-scale heterogeneous system; The optimization scheme is used to optimize the hardware and software of the large-scale heterogeneous system respectively to obtain an optimized large-scale heterogeneous system, wherein the performance of the software reliability assurance model when running in the optimized large-scale heterogeneous system is greater than the performance of the software reliability assurance model when running in the large-scale heterogeneous system before optimization.

7. The method according to claim 1, characterized in that Integrate the key performance indicators of multiple AI evaluation models and determine a unified evaluation standard, including: Obtaining key performance indicators of each of the artificial intelligence evaluation models, wherein the key performance indicators include at least one of the following: computing speed, efficiency, latency, throughput, model training, and throughput speed; Analyze the key performance indicators of each of the artificial intelligence evaluation models to determine a set of indicators that covers the key performance indicators of all of the artificial intelligence evaluation models; A unified evaluation standard is determined according to the indicator set.

8. An evaluation device for a software reliability assurance model in the electric power industry, characterized in that: include: A first processing unit is used to integrate the key performance indicators of multiple artificial intelligence evaluation models and determine a unified evaluation standard; A second processing unit is used to perform training using a historical data set of the power equipment and a target task scenario to obtain a software reliability assurance model, wherein the historical data set at least includes historical operating parameters of the software of the power equipment and historical performance parameters of the software; The third processing unit is used to repeatedly evaluate the performance of the software reliability assurance model based on the evaluation standard and using an integrated evaluation model to obtain multiple evaluation results, and calibrate the evaluation results using a cross-validation method to obtain a final evaluation result.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the evaluation method for the software reliability assurance model of the electric power industry as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include an evaluation method for executing the software reliability assurance model for the electric power industry as described in any one of claims 1 to 7.

Images