Multi-dimensional evaluation method and system for performance of power spot market solver

Abstract

The application provides a multi-dimensional evaluation method and system for the performance of a power spot market solver, and relates to the technical field of data analysis.The method comprises the following steps: constructing an SCUC simulation task set;building a containerized deployment multi-solver parallel solving architecture, accessing multiple solvers, and dividing the solvers into a first type of solver and a second type of solver, wherein the first type of solver is an unused solver, and the second type of solver is a used solver;respectively constructing an isolated first parallel solving running environment and a second parallel solving running environment;performing parallel testing to obtain a first group of performance test data sets and a second group of performance test data sets;performing multi-dimensional evaluation to output multiple performance evaluation indexes.The application solves the technical problem that the performance evaluation of the existing technology often depends on a single performance index, lacks multi-dimensional evaluation capability for the full generation cycle of the solver at different use stages, and leads to difficulty in guaranteeing the reliability of the solver in a complex power spot market environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, in particular to a multi-dimensional evaluation method and system for performance of a power spot market solver. BACKGROUND

[0002] The solver of the power spot market, especially in the application of security constrained unit commitment (SCUC) and security constrained economic dispatch (SCED), plays a crucial role. Their task is to dispatch and optimize according to multiple constraint conditions such as power demand, generator constraints, market rules, etc., to help the power system achieve the balance between economy and reliability. These solvers not only involve a large amount of data calculation, but also need to quickly give an effective dispatch scheme in a time-critical situation, so their solving speed, accuracy, stability, etc. directly affect the efficiency and safety of the power system. However, the performance evaluation of many existing solvers often only focuses on a single indicator such as solving time or optimal solution deviation, lacking a multi-dimensional performance evaluation system for the whole cycle of the solver product, thereby making it difficult to guarantee the stability and reliability of the solver in the complex power spot market environment, affecting the actual application effect. SUMMARY

[0003] The present application provides a multi-dimensional evaluation method and system for performance of a power spot market solver, aiming to solve the technical problem that the performance evaluation of the existing technology often relies on a single performance indicator, lacks multi-dimensional evaluation capability for the whole generation cycle of the solver in different use stages, and makes it difficult to guarantee the reliability of the solver in the complex power spot market environment, affecting the actual application effect.

[0004] The first aspect of the present application provides a multi-dimensional evaluation method for performance of a power spot market solver, the method comprising: constructing a SCUC simulation task set; building a containerized deployment multi-solver parallel solving architecture, accessing multiple solvers through a solver plug-in in the multi-solver parallel solving architecture, dividing the multiple solvers into a first type of solver and a second type of solver, wherein the first type of solver is an unused solver and the second type of solver is a used solver; constructing an isolated first parallel solving running environment and a second parallel solving running environment according to the first type of solver and the second type of solver; performing parallel testing on the first type of solver and the second type of solver according to the SCUC simulation task set in the first parallel solving running environment and the second parallel solving running environment, obtaining a first group of performance test data set and a second group of performance test data set; performing multi-dimensional evaluation on the first group of performance test data set and the second group of performance test data set, and outputting multiple performance evaluation indicators of the multiple solvers.

[0005] In a second aspect, the application discloses a multi-dimensional evaluation system for performance of a power spot market solver, which is used for the multi-dimensional evaluation method for performance of the power spot market solver, and comprises a task set construction module, a solver access module, a running environment construction module, a parallel test module and a multi-dimensional evaluation module.

[0006] The one or more technical solutions provided in the application have at least the following beneficial effects:

[0007] By constructing the SCUC simulation task set, a standardized task set is provided for performance testing of the solver, which simulates various scheduling tasks in the electricity spot market, can comprehensively evaluate the performance of the solver under different task conditions, and through this process, consistency and diversity of the solver can be ensured, thereby improving the reliability and comprehensiveness of the evaluation results; the multi-solver parallel solving architecture is built by using the containerized deployment method, and by plugging in multiple solvers through the solver plug-in, different solvers can run in parallel in the same architecture, and through the containerization technology, each solver can run in an isolated environment, avoiding interference between solvers and ensuring the independence and fairness of the test; the solvers are divided into the first type of solver and the second type of solver, and through this division, the performance difference between the newly connected solver and the used solver can be tested, ensuring the evaluation and verification of new products, and also monitoring the performance of existing products; by constructing the isolated first parallel solving running environment and the second parallel solving running environment, the test of each solver can be carried out in a completely independent environment, and this isolated design avoids resource competition and environmental interference between solvers, ensuring the accuracy of the test results; by testing the first type of solver and the second type of solver in the first parallel solving running environment and the second parallel solving running environment, the performance data of different types of solvers under multiple task scenarios are obtained, and the performance difference of different solvers under the same task set is quantified through the performance data set, providing specific basis for subsequent performance comparison; by multi-dimensional evaluation of the first group of performance test data set and the second group of performance test data set, the performance of the solver can be comprehensively evaluated, ensuring that the evaluation process not only depends on a single performance standard, but also integrates the important performance of the solver, and finally outputs multiple performance evaluation indexes of multiple solvers, making the evaluation results of the solver more intuitive and easy to understand, and providing scientific basis for subsequent solver selection, optimization and deployment.

[0008] The above description is only a summary of the technical solutions of the present application, in order to more clearly understand the technical means of the present application, the specific embodiments of the present application can be implemented in accordance with the content of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 The multi-dimensional evaluation method of the performance of the electricity spot market solver provided by the embodiment of the present application is shown in the flowchart.

[0010] Figure 2 The multi-dimensional evaluation system structure of the performance of the electricity spot market solver provided by the embodiment of the present application is shown in the schematic diagram.

[0011] Explanation of reference signs: task set construction module 10, solver access module 20, running environment construction module 30, parallel test module 40, and multi-dimensional evaluation module 50. DETAILED DESCRIPTION

[0012] The embodiments of the present application provide a multi-dimensional evaluation method and system for the performance of a power spot market solver, which solves the technical problem in the prior art that the performance evaluation of a solver often relies on a single performance indicator, lacks multi-dimensional evaluation capability for the full generation cycle of the solver in different use stages, and thus the reliability of the solver in a complex power spot market environment is difficult to guarantee, affecting the actual application effect.

[0013] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be specifically introduced below in combination with the drawings of the specification. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0014] In one embodiment, as shown in FIG. 1, the present application provides a multi-dimensional evaluation method for the performance of a power spot market solver, which comprises the following steps: Figure 1

[0015] Constructing a SCUC simulation task set.

[0016] The SCUC (Security-Constrained Unit Commitment) simulation task set refers to a set of unit scheduling tasks that simulate the operation of a power system. These unit scheduling tasks are used to evaluate the performance of a solver in actual power spot market operations. The SCUC simulation task set contains a variety of different tasks that involve different generating units, load demands, constraint conditions, and running time periods, covering different power spot market scenarios and complexities that may be encountered in actual operations. The data of the SCUC simulation task set can come from historical power spot market data, operation records of existing power systems, and forecasts of future power demand, fully reflecting the actual situation of the power spot market, including various operational constraints and unexpected events such as load fluctuations and generator failures.

[0017] A containerized deployment multi-solver parallel solving architecture is built, a plurality of solvers are accessed in the multi-solver parallel solving architecture through a solver plug-in, and the plurality of solvers are divided into a first type of solver and a second type of solver, wherein the first type of solver is an unused solver, and the second type of solver is a used solver.

[0018] ​The multi-solver parallel solving architecture is deployed using containerization technology, which allows different solvers to run in independent containers, thereby avoiding interference with each other, ensuring the isolation of tests and consistency of the environment, and the parallel solving architecture means that multiple solvers can run in parallel at the same time, thereby comparing their performance and improving solving efficiency. The multiple solvers are accessed through solver plug-ins, which means that different solvers can interact and communicate with the multi-solver parallel solving architecture through a unified plug-in interface. The solver plug-in is usually an abstract interface that allows different solvers to access the system in a standardized manner.

[0019] The multiple solvers are divided into two categories. The first category of solvers is unused solvers, i.e., solvers that have not been used or are new versions. These solvers are usually newly accessed solvers or solvers undergoing version upgrades. They have not been put into actual systems for use, but need to be tested and verified through simulation task sets. The characteristics include first deployment, no actual use, and no historical running data. The second category of solvers is used solvers, i.e., solvers that have been put into actual systems for use. These solvers have historical use records and running data. They are usually verified and stably running solvers. They are analyzed for performance at regular intervals after daily use. The characteristics include having been deployed and put into use, having historical running tasks and running data.

[0020] According to the first category of solvers and the second category of solvers, an isolated first parallel solving running environment and a second parallel solving running environment are respectively constructed.

[0021] A first parallel solving running environment is created for the first category of solvers, which is specifically used for testing new solvers. In this environment, the SCUC simulation task set is loaded, and the inputs and parameter configurations of the tasks are ensured to be suitable for processing by the new solvers. At the same time, a unified solving parameter template is set for the new solvers to ensure that their performance in the test meets the expected standards. Another second parallel solving running environment is created for the first category of solvers. This environment is relatively mature and can be optimized according to historical data and actual running conditions. Similarly, the same SCUC simulation task set is loaded, and a reasonable solving parameter template is set. Considering performance degradation and other issues, performance evaluation indicators such as solution quality and running time are additionally considered in this environment to comprehensively evaluate the solvers.

[0022] In the first parallel solving running environment and the second parallel solving running environment, the first category of solvers and the second category of solvers are tested in parallel according to the SCUC simulation task set, and a first set of performance test data and a second set of performance test data are obtained.

[0023] The first type of solver is started in a first parallel solving running environment, and the second type of solver is started in a second parallel solving running environment, and simulation tasks in the SCUC simulation task set are respectively executed, and in the process of parallel solving, each solver runs in an independent container to avoid mutual interference, and by collecting data of each solver in the running process, including solving time, resource consumption, solution quality, stability, etc., a corresponding first group of performance test data set and a second group of performance test data set are generated.

[0024] The first group of performance test data set and the second group of performance test data set are subjected to multi-dimensional evaluation, and a plurality of performance evaluation indexes of the plurality of solvers are output.

[0025] A multi-dimensional evaluation system is constructed, including threshold values of multi-dimensional preset performance evaluation indexes, which are used to quantify different performance dimensions of the solvers, including solving time, solution feasibility, solution optimality deviation and resource occupation fluctuation. The multi-dimensional preset performance evaluation indexes are used for multi-dimensional quantitative analysis of the first group of performance test data set and the second group of performance test data set, and the performance data of each solver in each dimension is extracted. According to the evaluation results of each dimension, single performance indexes of each solver are output. The evaluation results of each dimension are combined to obtain the comprehensive performance evaluation of each solver. The overall performance of different solvers in different tasks and scenes can be displayed by weighting the evaluation indexes of different dimensions and calculating the comprehensive score. On the basis of multi-dimensional evaluation, a plurality of performance evaluation indexes of a plurality of solvers are output.

[0026] Further, the plurality of solvers are divided into the first type of solver and the second type of solver, and the method comprises:

[0027] A solver information management table is constructed, which is used to store the life cycle state of the solver. The storage fields of the solver information management table include solver number, solver name, first registration time, deployment state, use state and historical running task set. A first determination condition is constructed. If the current solver satisfies any condition in the first determination condition, it is identified as the first type of solver. A second determination condition is constructed. If the current solver satisfies any condition in the second determination condition, it is identified as the second type of solver. The solver information management table is called to identify the plurality of solvers with the first determination condition and the second determination condition, and the first type of solver and the second type of solver are divided.

[0028] In the storage field of the solver information management table, the solver number is a unique number of each solver in the table, used to distinguish different solvers, which can be an automatically generated ID, or generated according to the specific rules of the solver (such as manufacturer number, version number); the solver name is the name of the solver, which is convenient for the administrator or developer to identify and find the solver, and the name is usually related to the model, version, manufacturer or function of the solver; the first registration time is the time when the solver is first registered in the system, that is, the registration time before the solver is first put into use, which is important for judging whether the solver is "first registered"; the deployment state is used to record whether the solver has been deployed in the actual system, if the solver has been running in the system, it is marked as deployed, if it has not been put into use, it is marked as not deployed; the use state is used to record whether the solver has been actually used, whether it has participated in the solving task, if the solver has participated in task scheduling or running, it is marked as used, if it has not participated in any task, it is marked as not used; the historical running task set is used to record all the tasks participated by the solver in history, these task records can show the actual running situation of the solver, including the type of task it solves, the running time, the result, etc., if the solver has no historical task, the field should be empty or have no record.

[0029] When a new solver is added to the system, the above information is timely recorded in the solver information management table, if the solver has been put into use, its deployment state, use state and historical task record are also updated according to the actual situation, with the life cycle of the solver advancing, such as deployment, use, participation in tasks, the related fields are updated, for example, when the solver is deployed to the actual environment, the deployment state should be updated to deployed, if the solver has executed a task, a new record is added to the historical task set, and the use state is marked as used.

[0030] The first determination condition is constructed, which includes that the solver is first registered, the solver is in an undeployed state, the solver is in an unused state, and the solver historical running task returns empty. If the current solver meets the above conditions, it is considered as the first type of solver, and the second type of solver is a newly connected, not widely used or in development solver.

[0031] The second determination condition is constructed, which is non-first registration, the solver is in a deployed state, the solver is in a used state, and the solver historical running task returns not empty. If the current solver meets the above conditions, it is considered as the second type of solver, and the second type of solver is a solver that has been put into production and has been actually tested, which has relatively mature performance and stability.

[0032] Detailed information of each solver is obtained from the solver information management table, including solver number, name, first registration time, deployment state, use state and historical running task set. According to the state and historical record of the solver, it is judged whether the solver meets the standard of the first type of solver. Specifically, if the first registration time of the solver is the earliest time in the current system and there is no other historical record, it is identified as the first type of solver; if the deployment state of the solver is not deployed, it is identified as the first type of solver; if the use state of the solver is not used, it is identified as the first type of solver; if the historical task record of the solver is empty, it means that it has not been actually used, and therefore it can also be identified as the first type of solver.

[0033] If the solver does not meet the first type of condition, it is checked whether it meets the condition of the second type of solver, i.e., if the registration time of the solver is earlier than the first registration record and it has been deployed, it is identified as the second type of solver; if the deployment state of the solver is deployed, it is identified as the second type of solver; if the use state of the solver is used, it is identified as the second type of solver; if there is a task record in the historical running task set of the solver, it means that it has been actually used, and therefore it is identified as the second type of solver.

[0034] Further, the first determination condition includes that the solver is first registered, the solver is in an undeployed state, the solver is in an unused state and the historical running task of the solver returns empty; and the second determination condition is non-first registration, the solver is in a deployed state, the solver is in a used state and the historical running task of the solver returns not empty.

[0035] In the first determination condition, first registration means that the solver is registered in the system for the first time, i.e., the system has no record of the solver before, and the first registered solver means that it is a new version of the solver or a completely new solver product; undeployed state means that the solver has not been actually deployed in the system for use, because the solver is still in the preparation stage or has only been tested in the development or test environment; unused state means that the solver has been registered but has not been used in actual tasks, and it can be a potential alternative solver that has not been actually run for verification; and the historical running task returns empty, which means that the solver has no historical running record, indicating that it has not been run in actual application or has been run but has no task execution.

[0036] In the second determination condition, the non-first registration means that the solver is not registered in the system for the first time, which means that it has been registered in the system, and it can have been used multiple times or upgraded; the deployed state means that the solver has been successfully deployed to the actual system, ready to be put into actual production environment or has started to be used; the used state means that the solver has been used multiple times in actual operation, indicating that it has been applied in actual scenarios and has stable operation records; the returned non-empty solver history operation task means that the solver has historical operation records, and the returned task set is not empty, which means that it has completed some tasks and accumulated performance data.

[0037] Further, according to the first type of solver and the second type of solver, an isolated first parallel solving operation environment and a second parallel solving operation environment are respectively constructed, and the method comprises:

[0038] initializing an isolated container; loading the SCUC simulation task set and the unified solving parameter template to deploy the initialized isolated container to obtain the first parallel solving operation environment; loading the SCUC simulation task set, the unified solving parameter template and the preset performance degradation evaluation index to deploy the initialized isolated container to obtain the second parallel solving operation environment.

[0039] The isolated container is initialized using containerization technology. Specifically, a suitable containerization platform such as a Docker container is selected. The Docker container can effectively provide isolation and efficient resource management, ensuring that the running environment of each solver does not interfere with other solvers, so that multiple parallel solvers can be tested in the same hardware environment. According to the test requirements, the resource limits of the isolated container, such as CPU and memory, are set. The configuration of the isolated container ensures that each solver can run in a controlled environment, preventing a certain solver from occupying too many resources and affecting the operation of other solvers.

[0040] The SCUC simulation task set contains various tasks for simulating power system scheduling; the unified solving parameter template includes the parameter settings required by all solvers, such as time limit, precision requirement, and resource allocation. The purpose of the template is to unify the solving parameter configuration, ensuring that the comparison of different solvers in the test process is carried out under the same conditions. The SCUC simulation task set and the unified solving parameter template are loaded into the initialized isolated container and start to be deployed to generate the first parallel solving operation environment, i.e., the standard test process environment. The standard test process refers to running a unified and standardized task test on each solver according to the SCUC simulation task set and the unified solving parameter template, ensuring consistency in comparison.

[0041] With the same standard test process, first load the SCUC simulation task set, unified solving parameter template, the difference is, also load the preset performance degradation evaluation index, which reflects the performance change trend of the solver in actual use, each performance degradation evaluation index interval includes the solution convergence time fluctuation range, resource occupation drift value, solution structure change rate. Deploy the above SCUC simulation task set, unified solving parameter template and preset performance degradation evaluation index to the initialization container to generate a second parallel solving running environment. This environment is different from the standard test process, focusing on the historical use and personalized review process of the solver to ensure the evaluation of its performance and potential degradation trend in long-term operation.

[0042] Further, after constructing the solver information management table, the storage information of each field in the solver information management table is analyzed to perform secondary division on the second type of solver, output a multi-level second type of solver, and construct a plurality of second parallel solving running environments according to the multi-level second type of solver; the multi-level second type of solver is tested in parallel in the plurality of second parallel solving running environments.

[0043] The solver information management table is analyzed, and a plurality of key feature vectors are read therefrom, including cumulative running time, cumulative task number, average single task time consumption, simulation task proportion, and fluctuation variance of performance evaluation index. A multi-level classification rule is defined according to the plurality of key feature vectors, such as a high-frequency use solver, a long-term running solver, and a recent abnormal solver. The second type of solver is divided into a plurality of sub-classes by the multi-level classification rule, and a multi-level second type of solver is output.

[0044] For the multi-level second type of solver, a plurality of second parallel solving running environments are constructed, such as a high-frequency use solver environment, a long-term running solver environment, and a recent abnormal solver environment. The high-frequency use solver environment is used to test the performance of the solver under a large number of tasks and complex constraint conditions. The long-term running solver environment is used to simulate the stability of the solver under long-term continuous operation and analyze whether the performance degrades. The recent abnormal solver environment is used to test how the solver responds to recent abnormalities and whether it can recover performance to avoid resource waste. In these different second parallel solving running environments, test tasks that meet the multi-level second type of solver are executed respectively, and parallel testing can speed up the testing efficiency and ensure performance evaluation under multiple scenarios.

[0045] Further, the storage information of each field in the solver information management table is analyzed to perform secondary division on the second type of solver, and a multi-level second type of solver is output, the method comprising:

[0046] According to the usage state in the solver information management table and the information of the historical running task set, a plurality of key feature vectors are collected, including cumulative running time, cumulative task number, average single task time consumption, simulation task proportion, and fluctuation variance of performance evaluation index; a multi-level classification rule is defined, and the second type of solver is divided again according to the multi-level classification rule, and a plurality of second type of solvers are output.

[0047] According to the field information in the solver information management table, the following plurality of key feature vectors are extracted: cumulative running time, indicating the total running time of the solver since the first use, a longer running time indicates that the solver has a more stable performance, or there is a performance degradation problem in long-term running; cumulative task number, indicating the number of tasks executed by the solver in history, a larger task quantity means that the solver has experienced a large amount of work load, which can reflect its performance under large-scale tasks; average single task time consumption, indicating the average solving time of each task, a longer single task time consumption indicates that the solver has a performance bottleneck; simulation task proportion, different types of tasks have different pressures and influences on the solver, by analyzing the proportion of task types, the performance of the solver in different types of tasks can be judged, such as high load tasks, complex constraint tasks, etc.; fluctuation variance of performance evaluation index, reflecting the performance fluctuation of the solver in different tasks, a larger fluctuation may imply the instability or performance degradation of the solver.

[0048] According to the above feature vectors, a multi-level classification rule is defined, for example, high-frequency use solver, long-term running solver, and recent abnormal solver, and the second type of solver is divided again by the multi-level classification rule, and a plurality of second type of solvers are output. Exemplarily, assuming that in the solver information management table, there is data such as Table 1: exemplary solver information management table, the second type of solver in the table is divided again, wherein SolverA_v1.0 belongs to high-frequency use solver (more task number) and long-term running solver (long running time); SolverB_v2.1 belongs to recent abnormal solver (although the running time is short, there may be abnormal performance fluctuation or resource occupation); SolverC_v3.0 belongs to long-term running solver, and may also be marked as high-load solver (due to high proportion of simulation tasks to high-load tasks).

[0049] Table 1 - Exemplary solver information management table

[0050]

[0051] Further, a plurality of second parallel solving running environments are constructed according to the plurality of second type of solvers, and the method comprises:

[0052] The SCUC simulation task set, the unified solution parameter template, and multiple preset performance degradation evaluation index intervals are loaded to deploy the initialized isolated container to obtain multiple second parallel solution running environments; wherein each performance degradation evaluation index interval includes the solution convergence time fluctuation range, the resource occupancy drift value, and the solution structure change rate.

[0053] The preset performance degradation evaluation index range is a standard for detecting whether the solver experiences performance degradation during operation. The design of these indicators can help determine whether the solver exhibits abnormal performance fluctuations or resource consumption. Each performance degradation evaluation index range includes the solution convergence time fluctuation range, resource occupancy drift value, and solution structure change rate. Among them, the solution convergence time fluctuation range is used to measure the degree of fluctuation in the time required for the solver to converge during the solution process. If the convergence time fluctuates greatly, it indicates that the solver has performance problems or instability in certain tasks; the resource occupancy drift value is used to measure the degree of fluctuation in the computing resources (such as CPU, memory) occupied by the solver when executing tasks. Excessive resource fluctuations or excessive resource occupation are manifestations of solver performance degradation; the solution structure change rate is used to evaluate the solution structure changes of the solver in multiple solutions. If the solution structure changes too much, it indicates that the solver cannot stably find the optimal solution, or there is an unreasonable calculation method.

[0054] The SCUC simulation task set, unified solution parameter template, and multiple preset performance degradation evaluation indicator intervals are loaded to deploy the initialized isolated container, creating multiple second parallel solution running environments for different types of solvers (such as high-frequency use, long-term operation, etc.). Each environment contains a SCUC simulation task set, unified solution parameter template, and preset performance degradation evaluation indicator intervals adapted to the characteristics of the corresponding solver, ensuring that the test can cover different usage scenarios of the solver and effectively evaluate its performance in long-term operation.

[0055] Furthermore, performing a multi-dimensional evaluation on the first set of performance test data sets and the second set of performance test data sets, and outputting a plurality of performance evaluation indicators of the plurality of solvers, the method includes:

[0056] Construct a multidimensional evaluation system, wherein the multidimensional evaluation system includes thresholds of multidimensional preset performance evaluation indicators, wherein the multidimensional preset performance evaluation indicators include solution time, solution feasibility, solution optimality deviation, and resource occupancy fluctuation; perform multidimensional evaluation on the first group of performance test data sets and the second group of performance test data sets according to the multidimensional evaluation system, and output the multidimensional performance evaluation indicators of each solver; calculate the multidimensional performance evaluation indicators of each solver by weight, and output multiple performance evaluation indicators of the multiple solvers.

[0057] A multi-dimensional evaluation system is constructed, which involves multi-dimensional preset performance evaluation indexes. These indexes can comprehensively reflect the overall performance of the solver. When establishing the multi-dimensional evaluation system, different threshold intervals are set for each index, which represent different levels of performance. Among them, the solving time is used to measure the time required for the solver to start solving to get the solution. For example, a solving time of less than 5 minutes indicates excellent performance, 5-10 minutes indicates medium performance, and more than 10 minutes indicates poor performance. The solution feasibility is used to evaluate whether the solver can meet all the constraints in the task and ensure that the output solution is legal. For example, a deviation of less than 5% is excellent, 5%-10% is medium, and more than 10% is poor. The solution optimality deviation is used to measure the deviation between the solution given by the solver and the theoretical optimal solution. A smaller deviation indicates that the solver can effectively find a solution close to the optimal solution. For example, a deviation of less than 5% is excellent, 5%-10% is medium, and more than 10% is poor. The resource occupancy fluctuation represents the stability of the resource (such as CPU, memory) occupancy of the solver when executing the task. If the resource usage fluctuation is large, it indicates that the efficiency of the solver is low or there are resource leaks, etc. For example, a fluctuation of less than 10% is excellent, 10%-20% is medium, and more than 20% is poor.

[0058] Using multi-dimensional preset performance evaluation indexes and their thresholds, the performance of each solver in the first and second sets of performance test data is evaluated in multiple dimensions. For solving time, the solving time of each solver in the test data set is used to determine whether it meets the preset threshold. For solution feasibility, the feasibility of each solver in solving the problem is checked to evaluate its feasibility. For solution optimality deviation, the deviation between the solution given by the solver and the theoretical optimal solution is calculated, and the performance is judged according to the preset threshold. For resource occupancy fluctuation, the resource occupancy fluctuation of each solver when executing the task is compared to determine the stability of its resource usage. According to the performance of each solver, the corresponding multi-dimensional performance evaluation indexes are output, which reflect the comprehensive performance of the solver in different tasks and environments.

[0059] According to the relative importance of each index, a weight is assigned to each index, and the allocation ratio is set according to actual needs. The multi-dimensional performance evaluation indexes of each solver are calculated by weighting, and the final performance evaluation index is the weighted sum of each index value and its corresponding weight. According to the calculation, the performance evaluation indexes of multiple solvers are output.

[0060] Further, the method comprises:

[0061] A reference solver is imported into the multi-solver parallel solving architecture, and the multi-dimensional performance index value of the reference solver is determined. Based on the multi-dimensional performance index value of the reference solver, the threshold of the multi-dimensional preset performance evaluation index of the multi-dimensional evaluation system is configured.

[0062] The reference solver can be a mature solver on the market, such as a representative solver that performs well in domestic solvers. The reference solver is connected to the multi-solver parallel solving architecture. The input task set and parameter template of the reference solver need to be consistent with other solvers, and ensure that the reference solver runs in the same environment as other solvers. The reference solver is comprehensively tested using the multi-dimensional evaluation system defined above, and its performance in each dimension is recorded. According to the test results, the multi-dimensional performance index value of the reference solver is determined, which provides benchmark data for subsequent solver evaluation.

[0063] Based on the multi-dimensional performance index value of the reference solver, a reasonable threshold is set for each evaluation dimension. For example, assuming that the solving time of the reference solver is 5 minutes, it can be set as the standard for excellent performance, and then the corresponding thresholds for excellent, medium and poor performance are set for other solvers. Assuming that the solution feasibility of the reference solver is 98%, the excellent standard of solution feasibility can be set to 95%, and the unqualified standard is less than 80%. Assuming that the optimality deviation of the reference solver is 3%, a deviation of less than 5% is set as excellent, and a deviation of more than 10% is set as unqualified. Assuming that the resource occupation fluctuation of the reference solver is 10%, a fluctuation of less than 15% is set as excellent, and a fluctuation of more than 20% is set as unqualified. The finally configured multi-dimensional evaluation system is output, which includes the thresholds of each performance dimension. These thresholds are used to evaluate the performance of other solvers.

[0064] In summary, the multi-dimensional evaluation method for the performance of the power spot market solver provided by the embodiments of the present application has the following technical effects:

[0065] By constructing the SCUC simulation task set, a standardized task set is provided for performance testing of solvers. This task set simulates various scheduling tasks in the electricity spot market and can comprehensively evaluate the performance of solvers under different task conditions. Through this process, the consistency and diversity of solver testing can be ensured, thereby improving the reliability and comprehensiveness of the evaluation results. A multi-solver parallel solution architecture is built using containerized deployment, and multiple solvers are connected through solver plug-ins, so that different solvers can run in parallel in the same architecture. Through containerization technology, each solver can run in an isolated environment, avoiding mutual interference between solvers and ensuring the independence and fairness of the test. Solvers are divided into first-class solvers and second-class solvers. Through this division, the performance difference between newly connected solvers and used solvers can be tested separately to ensure the evaluation and verification of new products, while also monitoring the performance of existing products. By constructing an isolated first parallel solver The test of each solver can be carried out in a completely independent environment. This isolated design avoids resource competition and environmental interference between solvers, and ensures the accuracy of the test results. By conducting parallel tests on the first and second types of solvers in the first and second parallel solving running environments, the performance data of different types of solvers in various task scenarios are obtained. The performance differences of different solvers under the same task set will be quantified through the performance data set, providing a specific basis for subsequent performance comparison. By performing multi-dimensional evaluation on the first and second performance test data sets, the performance of the solver can be comprehensively evaluated, ensuring that the evaluation process does not rely solely on a single performance standard, but integrates the various important performances of the solver, and finally outputs multiple performance evaluation indicators for multiple solvers, making the evaluation results of the solver more intuitive and easy to understand, providing a scientific basis for subsequent solver selection, optimization and deployment.

[0066] Example 2, based on the same inventive concept as the multi-dimensional evaluation method of the power spot market solver performance in the previous embodiment, such as Figure 2 As shown, an embodiment of the present application provides a multi-dimensional evaluation system for the performance of a power spot market solver, the system comprising:

[0067] The task set construction module 10 is used to construct the SCUC simulation task set.

[0068] The solver access module 20 is used to build a containerized multi-solver parallel solving architecture, connect multiple solvers to the multi-solver parallel solving architecture through the solver plug-in, and divide the multiple solvers into a first type of solvers and a second type of solvers, wherein the first type of solvers are unused solvers and the second type of solvers are used solvers.

[0069] An operation environment construction module 30 is configured to construct an isolated first parallel solving operation environment and an isolated second parallel solving operation environment according to the first-type solver and the second-type solver, respectively.

[0070] A parallel test module 40 is configured to perform parallel tests on the first-type solver and the second-type solver according to the SCUC simulation task set in the first parallel solving operation environment and the second parallel solving operation environment, and obtain a first group of performance test data set and a second group of performance test data set.

[0071] A multi-dimension evaluation module 50 is configured to perform multi-dimension evaluation on the first group of performance test data set and the second group of performance test data set, and output a plurality of performance evaluation indexes of the plurality of solvers.

[0072] Further, the solver access module 20 is configured to perform the following operation steps:

[0073] A solver information management table is constructed, and the solver information management table is configured to store the life cycle state of the solver, wherein the storage fields of the solver information management table include solver number, solver name, first registration time, deployment state, use state, and historical running task set; a first determination condition is constructed, and if the current solver satisfies any condition in the first determination condition, the current solver is identified as the first-type solver; a second determination condition is constructed, and if the current solver satisfies any condition in the second determination condition, the current solver is identified as the second-type solver; the plurality of solvers are identified by calling the solver information management table and the first determination condition and the second determination condition, and the first-type solver and the second-type solver are divided.

[0074] Further, the first determination condition includes that the solver is first registered, the solver is in an undeployed state, the solver is in an unused state, and the solver historical running task returns empty; and the second determination condition includes that the solver is not first registered, the solver is in a deployed state, the solver is in a used state, and the solver historical running task returns not empty.

[0075] Further, the operation environment construction module 30 is configured to perform the following operation steps:

[0076] An isolated container is initialized; the SCUC simulation task set and a unified solving parameter template are loaded to deploy the initialized isolated container to obtain the first parallel solving operation environment; the SCUC simulation task set, the unified solving parameter template, and a preset performance degradation evaluation index are loaded to deploy the initialized isolated container to obtain the second parallel solving operation environment.

[0077] Further, the solver access module 20 is configured to perform the following operation steps:

[0078] After the solver information management table is constructed, the storage information of each field in the solver information management table is parsed to perform secondary division on the second type of solver, a multi-level second type of solver is output, and a plurality of second parallel solver running environments are constructed according to the multi-level second type of solver; and the multi-level second type of solver is tested in parallel in the plurality of second parallel solver running environments.

[0079] Further, the solver access module 20 is configured to perform the following operation steps:

[0080] According to the information of the usage state and the historical running task set in the solver information management table, a plurality of key feature vectors are collected, the plurality of key feature vectors include cumulative running time, cumulative task number, average single task time consumption, simulation task proportion, and fluctuation variance of performance evaluation index; a multi-level classification rule is defined, the second type of solver is divided according to the multi-level classification rule, and a multi-level second type of solver is output.

[0081] Further, the solver access module 20 is configured to perform the following operation steps:

[0082] The SCUC simulation task set, the unified solver parameter template, and a plurality of preset performance degradation evaluation index intervals are loaded to deploy an initialized isolated container, and a plurality of second parallel solver running environments are obtained; wherein each performance degradation evaluation index interval includes a solution convergence time fluctuation range, a resource occupation drift value, and a solution structure change rate.

[0083] Further, the multi-dimensional evaluation module 50 is configured to perform the following operation steps:

[0084] A multi-dimensional evaluation system is constructed, the multi-dimensional evaluation system includes threshold values of a plurality of preset performance evaluation indexes, the plurality of preset performance evaluation indexes include solving time, solution feasibility, solution optimality deviation, and resource occupation fluctuation; the first group of performance test data set and the second group of performance test data set are evaluated in multi-dimensions according to the multi-dimensional evaluation system, and a plurality of performance evaluation indexes of each solver are output; the multi-dimensional performance evaluation indexes of each solver are weighted, and a plurality of performance evaluation indexes of the plurality of solvers are output.

[0085] Further, the multi-dimensional evaluation module 50 is configured to perform the following operation steps:

[0086] The reference solver is imported into the multi-solver parallel solving architecture, and the multi-dimensional performance index value of the reference solver is determined; based on the multi-dimensional performance index value of the reference solver, the threshold values of the multi-dimensional preset performance evaluation indexes of the multi-dimensional evaluation system are configured.

[0087] Through the foregoing detailed description of the method for multi-dimensionally evaluating the performance of the electricity spot market solver, those skilled in the art can clearly understand the multi-dimensionally evaluating system for the performance of the electricity spot market solver in the embodiment. Since the system corresponds to the method disclosed in the embodiment, the system is described simply, and the relevant part can be referred to the method part.

[0088] The foregoing description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Accordingly, the present application is not to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-dimensional evaluation method for the performance of a power spot market solver, characterized by: The method comprises: Constructing a SCUC simulation task set, wherein the SCUC simulation task set is a set of unit scheduling tasks simulating power system operation; Building a containerized multi-solver parallel solving architecture, connecting multiple solvers to the multi-solver parallel solving architecture through solver plug-ins, and dividing the multiple solvers into a first category of solvers and a second category of solvers, wherein the first category of solvers are unused solvers and the second category of solvers are used solvers; According to the first type of solvers and the second type of solvers, constructing an isolated first parallel solution execution environment and a second parallel solution execution environment respectively; In the first parallel solving operation environment and the second parallel solving operation environment, the first type of solver and the second type of solver are tested in parallel according to the SCUC simulation task set to obtain a first set of performance test data sets and a second set of performance test data sets; Perform a multi-dimensional evaluation on the first set of performance test data sets and the second set of performance test data sets, and output a plurality of performance evaluation indicators of the plurality of solvers.

2. The multi-dimensional evaluation method for the performance of the power spot market solver according to claim 1, characterized in that: Dividing the plurality of solvers into a first type of solvers and a second type of solvers, the method comprising: Constructing a solver information management table, wherein the solver information management table is used to store the life cycle status of the solver, wherein the storage fields of the solver information management table include the solver number, solver name, first registration time, deployment status, usage status, and historical running task set; Constructing a first determination condition, if the current solver meets any condition in the first determination condition, it is identified as a first-class solver; Constructing a second judgment condition, if the current solver meets any condition in the second judgment condition, it is identified as a second type of solver; The solver information management table is called to identify the plurality of solvers according to the first determination condition and the second determination condition, and to classify the solvers into first category and second category.

3. The multi-dimensional evaluation method for the performance of the power spot market solver according to claim 2, characterized in that: The first determination condition includes that the solver is registered for the first time, the solver is not deployed, the solver is not used, and the solver's historical running tasks return empty; The second judgment condition is that it is not the first registration, the solver is in a deployed state, the solver is in a used state, and the solver's historical running task return is not empty.

4. The multi-dimensional evaluation method for the performance of the power spot market solver according to claim 1, characterized in that: Based on the first type of solver and the second type of solver, an isolated first parallel solution execution environment and a second parallel solution execution environment are respectively constructed, the method comprising: Initialize the isolated container; Loading the SCUC simulation task set and the unified solution parameter template to deploy the initialized isolation container to obtain a first parallel solution running environment, wherein the SCUC simulation task set is a set of unit scheduling tasks in simulating power system operation; The SCUC simulation task set, the unified solution parameter template, and the preset performance degradation evaluation index are loaded to deploy the initialized isolation container to obtain a second parallel solution running environment.

5. The multi-dimensional evaluation method for the performance of the power spot market solver according to claim 2, characterized in that: After constructing the solver information management table, parsing the storage information of each field in the solver information management table to perform secondary division on the second type of solvers, outputting a multi-level second type of solvers, and constructing a plurality of second parallel solver operating environments based on the multi-level second type of solvers; The multi-level second-type solvers are tested in parallel in the multiple second parallel solving operating environments.

6. The multi-dimensional evaluation method for the performance of the power spot market solver according to claim 5, characterized in that: Parsing the storage information of each field in the solver information management table to perform secondary division on the second type of solvers, and outputting multi-level second type solvers, the method includes: Collect multiple key feature vectors based on the usage status and historical running task set information in the solver information management table, wherein the multiple key feature vectors include cumulative running time, cumulative number of tasks, average single task time, simulation task ratio, and fluctuation variance of performance evaluation indicators; A multi-level classification rule is defined, the second-type solvers are divided twice according to the multi-level classification rule, and a multi-level second-type solver is output.

7. The multi-dimensional evaluation method for the performance of the power spot market solver according to claim 5, characterized in that: Constructing multiple second parallel solution execution environments according to the multi-level second-type solvers, the method includes: Loading the SCUC simulation task set, a unified solution parameter template, and multiple preset performance degradation evaluation index intervals to deploy the initialized isolation container to obtain multiple second parallel solution operation environments, wherein the SCUC simulation task set is a set of unit scheduling tasks in simulating power system operation; Among them, each performance degradation evaluation index interval includes the solution convergence time fluctuation range, resource occupancy drift value, and solution structure change rate.

8. The multi-dimensional evaluation method for the performance of a power spot market solver according to claim 1, wherein: Performing a multi-dimensional evaluation on the first set of performance test data sets and the second set of performance test data sets, and outputting a plurality of performance evaluation indicators of the plurality of solvers, the method comprising: Constructing a multi-dimensional evaluation system, the multi-dimensional evaluation system includes thresholds for multi-dimensional preset performance evaluation indicators, the multi-dimensional preset performance evaluation indicators including solution time, solution feasibility, solution optimality deviation, and resource occupancy fluctuation; Performing a multi-dimensional evaluation on the first set of performance test data sets and the second set of performance test data sets according to the multi-dimensional evaluation system, and outputting a multi-dimensional performance evaluation index for each solver; The multi-dimensional performance evaluation index of each solver is calculated by weight, and multiple performance evaluation indexes of the multiple solvers are output.

9. The multi-dimensional evaluation method for the performance of the power spot market solver according to claim 8, characterized in that: The method comprises: Importing a reference solver into the multi-solver parallel solving architecture and determining multi-dimensional performance index values ​​of the reference solver; Based on the multi-dimensional performance indicator values ​​of the reference solver, thresholds of the multi-dimensional preset performance evaluation indicators of the multi-dimensional evaluation system are configured.

10. A multi-dimensional evaluation system for the performance of electricity spot market solvers, characterized by: A multi-dimensional evaluation method for the performance of a power spot market solver according to any one of claims 1 to 9, the system comprising: A task set construction module is used to construct a SCUC simulation task set, wherein the SCUC simulation task set is a set of unit scheduling tasks in simulating power system operation; A solver access module is used to build a containerized multi-solver parallel solution architecture, connect multiple solvers to the multi-solver parallel solution architecture through a solver plug-in, and divide the multiple solvers into a first type of solvers and a second type of solvers, wherein the first type of solvers are unused solvers and the second type of solvers are used solvers; An operating environment construction module, configured to construct an isolated first parallel solution operating environment and a second parallel solution operating environment according to the first type of solver and the second type of solver, respectively; A parallel testing module is used to perform parallel testing on the first type of solver and the second type of solver according to the SCUC simulation task set in the first parallel solving running environment and the second parallel solving running environment, to obtain a first set of performance test data sets and a second set of performance test data sets; A multi-dimensional evaluation module is used to perform multi-dimensional evaluation on the first group of performance test data sets and the second group of performance test data sets, and output multiple performance evaluation indicators of the multiple solvers.

Images