A power collaborative scheduling evaluation method based on reinforcement learning

Abstract

This invention relates to the field of power computing power collaborative optimization and intelligent scheduling technology, specifically a power collaborative scheduling evaluation method based on reinforcement learning. This method constructs a microgrid scheduling model with the goal of minimizing overall operating costs, encompassing distributed power generation costs, energy storage losses, grid interaction, and power shortage penalties. It also rigorously embeds dynamic constraints on energy storage charging and discharging power, capacity, and SOC. By introducing an energy storage constraint adaptation factor α, the method improves the bee foraging algorithm and establishes a mapping relationship between nectar sources and energy storage states. The fitness value is calculated by weighting the objective function and constraint satisfaction. The neighbor search step size of the hired bees is dynamically adjusted with α, and scout bees generate new nectar sources based on constraint boundaries. Combined with a dynamic crossover and mutation operator, local optima are effectively avoided. The final output is an optimal scheduling scheme that includes photovoltaic, wind power, diesel engine output, and energy storage charging and discharging strategies, suitable for intelligent scheduling of microgrids with a high proportion of renewable energy.

Description

Technical Field

[0001] This invention relates to the field of power computing power collaborative optimization and intelligent scheduling technology, specifically a power collaborative scheduling evaluation method based on reinforcement learning. Background Technology

[0002] Currently, power computing collaborative systems widely utilize reinforcement learning algorithms for resource scheduling. By analyzing hardware instruction set parameters and task metadata, they optimize energy consumption and efficiency, attempting to find the optimal scheduling strategy in a dynamic environment. However, with the increasing complexity of the system's operating environment, when a decline in energy efficiency is detected, existing assessment methods struggle to effectively distinguish whether the anomaly is caused by the exploratory trial and error of the reinforcement learning scheduling strategy itself, or by external environmental noise, such as sensor overshoot or measurement noise interference. Traditional methods often use historical averages as the assessment benchmark, lacking a physical rigid reference system independent of the scheduling algorithm and environmental factors. This can easily lead to misjudging long-term inefficient operation as normal, and it is difficult to capture minute deviations such as millisecond-level task switching delays. This lack of identification capability prevents the operation and maintenance system from accurately locating the internal cause of failure. Blindly responding to environmental noise can easily cause scheduling strategy oscillations, resulting in insufficient long-term system stability. Therefore, there is an urgent need for an assessment scheme that can establish a theoretical benchmark based on physical limits and accurately decouple strategic failures from environmental noise from a mathematical perspective, in order to solve the problems of misjudgment risk and lack of basis for targeted optimization in existing technologies. Summary of the Invention

[0003] The purpose of this invention is to provide a power cooperative scheduling evaluation method based on reinforcement learning. This method can establish a theoretical benchmark independent of the scheduling strategy based on physical limits to eliminate evaluation bias, and can accurately decouple strategic failures from environmental noise at the mathematical level, effectively avoiding scheduling strategy oscillations caused by blindly responding to environmental noise. Specifically, the technical solution of this invention is as follows: S1. Collect hardware instruction set parameters and task metadata from the power computing collaborative system, establish a theoretical minimum energy consumption model, and generate an ideal baseline state vector by combining real-time electricity price data and carbon intensity data. S2. Construct a fault feature direction vector based on the scheduling failure mode, and combine it with the interference intensity coefficient to generate a simulated disturbed state vector. S3. Collect real-time running data, calculate the difference between the real-time running data and the ideal baseline state vector, and generate the actual residual vector; S4. Calculate the difference between the simulated disturbed state vector and the ideal reference state vector to generate the theoretical residual vector; S5. Map the real residual vector and the theoretical residual vector to a high-dimensional feature space, calculate the geometric overlap, and generate the coupling coefficient. S6. Compare the coupling coefficient with the preset confidence threshold. If the coupling coefficient is greater than the confidence threshold, it is determined that the energy efficiency decline is caused by the failure of the scheduling strategy, and the first judgment result is output. If the coupling coefficient is less than or equal to the confidence threshold, it is determined that the energy efficiency decline is caused by environmental noise, and the second judgment result is output.

[0004] Preferably, S1 includes: S11. By analyzing the hardware instruction set architecture, obtain the physical limit energy consumption data of the chip when executing computationally intensive and memory-intensive operators; S12. Based on physical limit energy consumption data, establish a theoretical minimum energy consumption model that is independent of scheduling strategies and environmental noise; S13. Based on real-time electricity price data and carbon intensity data, use linear programming to calculate the minimum cost and minimum carbon emission trajectory; S14. Combining the theoretical minimum energy consumption model, minimum cost, and minimum carbon emission trajectory, generate an ideal baseline state vector.

[0005] Preferably, S1 further includes: S15. Perform differential statistics through the instruction flow monitoring window to obtain the instantaneous execution rate of the operator; S16. Calculate the dynamic power consumption of the computing core based on the instantaneous execution rate and the physical limit energy consumption of a single operation of the operator; S17. Obtain the instantaneous data throughput of the task through bandwidth monitoring data; S18. Calculate the transmission power consumption of the subsystem based on the instantaneous data throughput and the energy consumption limit of physical transmission per unit of data. S19. Sum the chip's static power consumption, the calculation core's dynamic power consumption, and the subsystem's transmission power consumption to generate the instantaneous theoretical minimum power, and construct an ideal reference state vector based on the instantaneous theoretical minimum power.

[0006] Preferably, S2 includes: S21. Based on historical fault data, extract features of reinforcement learning scheduling failure modes and construct fault feature direction vectors; wherein, the fault feature direction vectors are normalized. S22. Determine the interference intensity coefficient based on the current system load level or historical case fitting; S23. Calculate the product of the fault characteristic direction vector and the interference intensity coefficient to generate the interference operator; S24. Superimpose the disturbance operator onto the ideal reference state vector to generate the simulated disturbed state vector.

[0007] Preferably, S5 includes: S51. Using the cosine similarity algorithm, calculate the dot product between the actual residual vector and the theoretical residual vector; S52. Calculate the product of the magnitude of the actual residual vector and the magnitude of the theoretical residual vector; S53. Compare the product with the numerically stable minimum value and select the larger value as the denominator. S54. Divide the dot product by the denominator to generate the coupling coefficient.

[0008] Preferably, the method further includes the step of determining a confidence threshold: S01. Obtain the historical labeled dataset and iterate through different candidate thresholds; S02. For each candidate threshold, calculate the harmonic mean of precision and recall to generate an F1 score; S03. Select the candidate threshold that maximizes the F1 score and determine the candidate threshold as the confidence threshold.

[0009] Preferably, it also includes a feedback step based on the determination result: S71. In response to the first decision result, identify the reinforcement learning strategy pattern that leads to failure; S72, triggers parameter adjustment or retraining processes for reinforcement learning policy patterns; S73. In response to the second determination result, maintain the current scheduling strategy and record the environmental noise characteristics. Compared with the prior art, the present invention has the following beneficial effects: 1. This invention introduces a dual-track comparison mechanism of ideal benchmark and simulated disturbed state to construct real residual and theoretical residual vectors, and calculates geometric overlap in normalized feature space; this method successfully decouples strategic failures and environmental noise mixed in the running data mathematically, solves the problem that existing technologies cannot distinguish whether energy efficiency degradation is caused by scheduling trial and error or external interference, and achieves accurate identification of anomaly sources; 2. This invention obtains physical limit energy consumption data based on hardware instruction set architecture analysis and establishes a theoretical minimum energy consumption model independent of scheduling strategies and environmental noise. This bottom-up physical modeling method establishes an absolute reference system for evaluation, eliminates the bias caused by the traditional method of using historical averages as a benchmark, effectively prevents the misjudgment of long-term inefficient operation as a normal state, and ensures the objectivity and physical rigidity of the evaluation benchmark. 3. This invention performs differential statistics and bandwidth monitoring through the instruction stream monitoring window, and combines the physical limit energy consumption of the operator to generate the instantaneous theoretical minimum power with microsecond-level resolution. This high-granularity evaluation method can accurately capture tiny scheduling strategy deviations such as millisecond-level task switching delays, significantly improving the ability to perceive instantaneous energy efficiency anomalies in dynamic environments and making up for the shortcomings of existing technologies in fine evaluation. 4. This invention constructs a feedback closed loop based on the judgment result, which triggers parameter adjustment or retraining for policy failure, and maintains the current policy and records features for environmental noise. This differentiated response mechanism clarifies the optimization direction, effectively avoids scheduling policy oscillations caused by blindly responding to environmental noise, provides an interpretable basis for the targeted optimization of reinforcement learning models, and significantly improves the long-term operational stability of the system. Attached Figure Description

[0010] The present invention will be further explained below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0012] Example 1: Please see Figure 1 A power cooperative scheduling evaluation method based on reinforcement learning, the specific steps of which include: S1. Collect hardware instruction set parameters and task metadata from the power computing collaborative system, establish a theoretical minimum energy consumption model, and generate an ideal baseline state vector by combining real-time electricity price data and carbon intensity data. S2. Construct a fault feature direction vector based on the scheduling failure mode, and combine it with the interference intensity coefficient to generate a simulated disturbed state vector. S3. Collect real-time running data, calculate the difference between the real-time running data and the ideal baseline state vector, and generate the actual residual vector; S4. Calculate the difference between the simulated disturbed state vector and the ideal reference state vector to generate the theoretical residual vector; S5. Directly calculate the geometric overlap between the actual residual vector and the theoretical residual vector in the normalized feature space to generate the coupling coefficient. S6. Compare the coupling coefficient with the preset confidence threshold. If the coupling coefficient is greater than the confidence threshold, it is determined that the energy efficiency decline is caused by the failure of the scheduling strategy, and the first judgment result is output. If the coupling coefficient is less than or equal to the confidence threshold, it is determined that the energy efficiency decline is caused by environmental noise, and the second judgment result is output.

[0013] This embodiment provides a power cooperative scheduling evaluation method based on reinforcement learning. The method aims to solve the problem in the prior art that it is difficult to distinguish whether the energy efficiency degradation of the power computing power cooperative system is caused by the exploratory trial and error of the reinforcement learning scheduling strategy itself or by external environmental noise. The method in this embodiment collects hardware instruction set parameters and task metadata from the power computing collaborative system. It analyzes the hardware instruction set architecture to obtain physical limit energy consumption data when the chip executes different types of operators. Based on this physical limit energy consumption data, a theoretical minimum energy consumption model independent of scheduling strategies and environmental noise is established. Combining real-time electricity price data and carbon intensity data, a linear programming method is used to calculate the minimum cost and minimum carbon emission trajectory, thereby generating an ideal baseline state vector. The original data for each component of the ideal reference state vector includes physical quantities such as the theoretical minimum power and the theoretical minimum delay. It is worth noting that, since power and delay belong to different physical dimensions and have vastly different numerical magnitudes, this embodiment performs dimensionless mapping processing on the aforementioned physical quantities to ensure the mathematical validity of subsequent geometric distance and angle calculations in the feature space. Specifically, the system presets normalization benchmark parameters such as the chip's TDP power consumption limit and maximum tolerable delay. To eliminate the influence of sensor instantaneous overshoot or measurement noise on feature space calculations, the system truncates abnormal measurement values ​​that exceed the preset physical limits, ensuring that the values ​​are strictly limited within the physical boundaries. The system uses maximum-minimum normalization or Z-score normalization methods to map the original values ​​of each physical quantity to a preset dimensionless interval or standard normal distribution space. The final generated ideal benchmark state vector is a multi-dimensional dimensionless feature vector, where each dimension component only represents the normalization strength of the attribute relative to the benchmark, thus establishing a purely mathematical reference system independent of physical units. As an absolute reference system for evaluating the actual operating state, it is completely independent of the current scheduling strategy. This embodiment constructs a fault feature direction vector based on scheduling failure modes and combines it with the interference intensity coefficient to generate a simulated disturbed state vector. This step applies counterfactual reasoning logic, that is, simulating the state the system should exhibit if a specific scheduling failure occurs; the system collects real-time operational data. The difference between the real-time running data and the ideal baseline state vector is calculated to generate a real residual vector that represents the total deviation observed in the real world. Simultaneously, the difference between the simulated disturbed state vector and the ideal reference state vector is calculated to generate a theoretical residual vector that characterizes the theoretical deviation fingerprint caused purely by a specific fault mode. ; Based on this, this embodiment maps the actual residual vector and the theoretical residual vector to the same high-dimensional feature space, and calculates their geometric overlap to generate a coupling coefficient. Compare the coupling coefficient with a preset confidence threshold; if the coupling coefficient is greater than the confidence threshold... If the energy efficiency degradation is determined to be caused by the failure of the scheduling strategy, the first determination result is output; if the coupling coefficient is less than or equal to the confidence threshold, the first determination result is output. If the energy efficiency decline is determined to be caused by environmental noise, a second determination result is output. By introducing a dual-track comparison mechanism of ideal benchmark and simulated disturbed state, this application successfully decouples strategic failures and environmental noise mixed in the operating data mathematically. It can not only identify energy efficiency anomalies, but also accurately identify specific strategic problems through geometric overlap, thus providing an interpretable basis for the targeted optimization of reinforcement learning models.

[0014] Example 2: S1 includes: S11. By analyzing the hardware instruction set architecture, obtain the physical limit energy consumption data of the chip when executing computationally intensive and memory-intensive operators; S12. Based on physical limit energy consumption data, establish a theoretical minimum energy consumption model that is independent of scheduling strategies and environmental noise; S13. Based on real-time electricity price data and carbon intensity data, use linear programming to calculate the minimum cost and minimum carbon emission trajectory; S14. Combining the theoretical minimum energy consumption model, minimum cost, and minimum carbon emission trajectory, generate an ideal baseline state vector.

[0015] This embodiment refines the process of establishing the ideal baseline state vector at the physical level. To ensure the objectivity of the baseline, this embodiment adopts a bottom-up physical modeling method. Specifically, through hardware instruction set architecture, i.e., ISA analysis, physical limit energy consumption data of a specific chip is obtained for computationally intensive operators such as matrix multiplication and storage-intensive operators such as large memory copy. This physical limit energy consumption data refers to the minimum energy consumption of the chip when executing a specific instruction, which comes from the ISA energy efficiency fingerprint library provided by the chip manufacturer or is determined through micro-benchmark testing. Based on the above physical limit energy consumption data, a theoretical minimum energy consumption model is established. This model is only related to the computational and data volume of the task itself, and eliminates the influence of environmental factors such as the selection of scheduling algorithms or the temperature of the computer room air conditioning. This embodiment introduces real-time electricity price data and carbon intensity data as external constraints, transforming the task's data dependency graph into topological time-series constraints. A mixed-integer linear programming method is employed to calculate the minimum cost and minimum carbon emission trajectory while satisfying the task deadline and dependency order. The theoretical minimum energy consumption model is combined with the minimum cost and minimum carbon emission trajectory to generate an ideal baseline state vector containing a time dimension. By constructing a theoretical minimum energy consumption model at the instruction set architecture level, this embodiment establishes an absolute zero point independent of any scheduling algorithm, eliminating the bias caused by using historical averages as a benchmark in traditional methods, preventing the misjudgment of long-term inefficient operation as a normal state, and ensuring the physical rigidity of the evaluation benchmark.

[0016] Example 3: S1 also includes: S15. Perform differential statistics through the instruction flow monitoring window to obtain the instantaneous execution rate of the operator; S16. Calculate the dynamic power consumption of the computing core based on the instantaneous execution rate and the physical limit energy consumption of a single operation of the operator; S17. Obtain the instantaneous data throughput of the task through bandwidth monitoring data; S18. Calculate the transmission power consumption of the subsystem based on the instantaneous data throughput and the energy consumption limit of physical transmission per unit of data. S19. Sum the chip's static power consumption, the calculation core's dynamic power consumption, and the subsystem's transmission power consumption to generate the instantaneous theoretical minimum power, and construct an ideal reference state vector based on the instantaneous theoretical minimum power.

[0017] This embodiment further refines the construction process of the ideal baseline state vector, particularly regarding how to transform static energy consumption data into dynamic instantaneous theoretical minimum power with microsecond-level resolution. This embodiment uses an instruction stream monitoring window to perform differential statistics to obtain the instantaneous execution rate of the operator. Based on this instantaneous execution rate and the physical limit energy consumption of a single operator operation, the dynamic power consumption of the computational core is calculated. Simultaneously, the instantaneous data throughput of the task is obtained through bandwidth monitoring data, and combined with the unit data physical transmission energy consumption limit, the subsystem transmission power consumption is calculated. This unit data physical transmission energy consumption limit originates from hardware bus protocol standards such as the PCIe specification or physical layer electrical parameters. The chip's basic static power consumption, the computing core's dynamic power consumption, and the subsystem's transmission power consumption are summed to generate... The original value of the instantaneous theoretical minimum power at time t. And by performing dimensionless processing, we obtain Specifically, this calculation and normalization process is obtained through the following formula: in, This represents the chip's static leakage current power consumption, which is a constant. This indicates the total number of operators or task types supported by the system. Indicates the first The instantaneous execution rate of the operator, measured in times per second, is obtained by differential statistics from the instruction stream monitoring window of the job scheduler. Indicates the first The physical limit energy consumption of a single operation of the operator, in joules per operation, is derived from a pre-built chip ISA energy efficiency fingerprint database. Indicates the first The instantaneous data throughput of the task is measured in bits per second and is derived from memory bus or network interface bandwidth monitoring data after deduplication and smoothing. This represents the physical energy consumption limit for transmitting a unit of data, expressed in joules per bit. This represents the upper limit of the chip's thermal design power, expressed in watts. At this point, the original value of the instantaneous theoretical minimum power is calculated. : Utilizing the upper limit of chip thermal design power Normalize it to generate dimensionless power components. The calculation formula is as follows: If the calculation result is greater than 1, then take... To eliminate the effects of instantaneous overshoot; The normalized dimensionless power component, together with the dimensionless delay component generated in subsequent steps, constitutes the ideal reference state vector. Its value only represents the normalization intensity relative to the thermal design power; it is a dimensionless value; the trajectory equation proposed in this embodiment can accurately depict the shape of the power curve under perfect scheduling. By introducing differential statistics and instantaneous throughput, this model provides microsecond-level time resolution, enabling subsequent steps to capture minute scheduling policy deviations such as millisecond-level task switching delays, significantly improving the granularity of the evaluation; in addition, for the delay component in the ideal baseline state vector This embodiment obtains the critical path based on dynamic critical path analysis of the task dependency graph (DAG). Specifically, the system analyzes the topology of active tasks within the current time window in real time and identifies the critical path with the longest theoretical execution time. And calculate the theoretical minimum delay according to the following formula: in, express The set of critical path nodes in the dependency graph of tasks that are not yet completed at any given time; For nodes The number of floating-point operations; This represents the chip's theoretical peak computing performance. For nodes The amount of data to be transmitted; This represents the physical bandwidth limit of the hardware bus. The maximum tolerable delay allowed by the pre-defined Service Level Agreement (SLA) right Normalization is performed to generate a dimensionless delay component: Build Instantaneous reference state vector at time t This dual normalization based on physical limits and SLA constraints ensures the physical integrity and numerical comparability of the ideal reference state vector in both power and delay dimensions.

[0018] Example 4: S2 includes: S21. Based on historical fault data, extract features of reinforcement learning scheduling failure modes and construct fault feature direction vectors; wherein, the fault feature direction vectors are normalized. S22. Determine the interference intensity coefficient based on the current system load level or historical case fitting; S23. Calculate the product of the fault characteristic direction vector and the interference intensity coefficient to generate the interference operator; S24. Superimpose the disturbance operator onto the ideal reference state vector to generate the simulated disturbed state vector.

[0019] This embodiment details the process of generating simulated disturbed state vectors through parameterization, i.e., how to transform abstract fault knowledge into computable vectors; and how to extract features of reinforcement learning scheduling failure modes based on historical fault data to construct fault feature direction vectors. The fault feature direction vector is normalized so that its magnitude is equal to 1, thus representing only the direction of the fault in the multidimensional space, such as the specific proportional relationship of a sharp increase in power and a slight increase in delay; this vector is constructed by the expert system based on the feature extraction of historical fault data, such as the pre-defined direction of PCA principal components. The interference strength coefficient is determined by adaptive scaling based on the current system load level or by fitting historical cases. The specific adaptive computation function is a piecewise nonlinear function, where... The load percentage values ​​range from 0 to 100: in, The preset intensity gain factor is taken in this embodiment. The quadratic term in this formula reflects the nonlinear amplification characteristic, that is, when the system load... Near full load, even minor fault disturbances can lead to drastic deviations from the system state, consistent with the physical law that system vulnerability increases under high load. This formula indicates that when the system is operating under high load, the disturbance intensity of a fault is nonlinearly amplified. The product of the fault characteristic direction vector and the disturbance intensity coefficient is calculated to generate a disturbance operator, which is then superimposed onto the ideal reference state vector. It should be noted that the fault characteristic direction vector... It is a normalized two-dimensional constant vector ,satisfy This characterizes the proportion of the fault type on the power axis and delay axis; when generating the simulated disturbed state vector, this constant vector is superimposed onto the time-varying ideal reference trajectory through a broadcast mechanism. The specific calculation formula is as follows: in, for Simulated disturbed state vector at time step; by using static structured fault features Dynamically superimposed onto an ideal physical reference In this study, the method generates a dynamic simulation trajectory that includes both the physical fluctuation characteristics of the task itself and the specific fault deviation characteristics. This generates the simulated disturbed state vector. The specific calculation formula is as follows: in, for The simulated disturbed state vector at each time step; by superimposing the static fault characteristic direction onto the dynamic reference trajectory at each time step, the continuous impact of persistent faults on the system state is simulated; due to the ideal reference state vector With fault feature direction vector All are already in the dimensionless characteristic space, therefore the interference intensity coefficient Defined as a dimensionless scalar scaling factor; this coefficient is used only to characterize the relative severity of the fault deviation from the baseline; This indicates moderate-intensity interference. This method represents severe disturbances without requiring specific physical units. By superimposing vectors in a dimensionless space, this embodiment not only solves the technical problem of different physical quantities not being directly summable, but also achieves mathematical decoupling between fault characteristics and the baseline state. This method ensures that the generated simulated disturbed state retains the structured characteristics of the fault, with the direction shifting from... The decision was made, and it was adapted to the current interference intensity. Decide.

[0020] Example 5: S5 includes: S51. Using the cosine similarity algorithm, calculate the dot product between the actual residual vector and the theoretical residual vector; S52. Calculate the product of the magnitude of the actual residual vector and the magnitude of the theoretical residual vector; S53. Compare the product with the numerically stable minimum value and select the larger value as the denominator. S54. Divide the dot product by the denominator to generate the coupling coefficient.

[0021] This embodiment details a method for calculating geometric overlap in a high-dimensional feature space. In order to effectively filter out random noise by utilizing the sparse orthogonality of the high-dimensional space, this embodiment does not calculate based on vectors at a single point in time, but rather constructs a high-dimensional feature tensor based on a time sliding window. The specific steps are as follows: S51, Define the time window length For example, take Each sampling period extracts the current time. and the past The residual data at each time step are used to flatten the two-dimensional instantaneous residual vector sequence, constructing a dimension of... High-dimensional feature vectors: Reality residual high-dimensional vector Theoretical residual high-dimensional vector in, and Both are two-dimensional vectors containing power and delay components, therefore and The total dimension is ; S52. Calculate high-dimensional vectors using the cosine similarity algorithm. and The dot product; S53, Calculation model and The product of modulo; S54, Combine the product with the numerically stable minimum. Comparison, for example, taking a value Choose a larger value as the denominator to prevent the denominator from being zero; S55. Divide the dot product by the denominator to generate the coupling coefficient. The specific calculation formula is revised as follows: Through In a 3D feature space, the cumulative dot product of a random environmental noise vector following a Gaussian distribution approaches zero within a time window, while scheduling faults with specific directional characteristics produce significant cumulative projections. Therefore, only when the real-world data truly contains features highly consistent with the theoretical fault modes over time series... Only then will it significantly approach 1; in, , ; For example, a preset system constant, with a value of This formula utilizes the mathematical property that Gaussian white noise and structured features are usually orthogonal in high-dimensional space. Through dot product operations, randomly distributed environmental noise in the high-dimensional feature space is automatically filtered out, only when the real data actually contains scheduling fault features consistent with the direction. Only then will it significantly approach 1; introduce a numerically stable minimum. This greatly enhances the numerical stability of the algorithm during stable system operation and avoids singular value errors.

[0022] Example 6: This method also includes the step of determining the confidence threshold: S01. Obtain the historical labeled dataset and iterate through different candidate thresholds; S02. For each candidate threshold, calculate the harmonic mean of precision and recall to generate an F1 score; S03. Select the candidate threshold that maximizes the F1 score and determine the candidate threshold as the confidence threshold.

[0023] This embodiment describes the specific steps for determining the confidence threshold. To balance false positives and false negatives, this embodiment abandons the practice of setting the threshold based on experience and instead adopts a statistical method. Specifically, it obtains a historical labeled dataset containing labeled fault types and noise types. Iterate through different candidate thresholds within the range of 0 to 1. For each candidate threshold Calculate the precision and recall based on the classification results at this threshold, and then calculate the harmonic mean of the two to generate the F1 score. ; Finally, the candidate threshold that maximizes the F1 score is selected. The candidate threshold is then determined as the confidence threshold for system operation. This process ensures that the evaluation system maintains high sensitivity, i.e., does not miss subtle policy failures, while minimizing the false alarm rate, i.e., does not mistake random noise for faults, thus achieving statistically optimal decision-making.

[0024] Example 7: This method also includes a feedback step based on the determination result: S71. In response to the first decision result, identify the reinforcement learning strategy pattern that leads to failure; S72, triggers parameter adjustment or retraining processes for reinforcement learning policy patterns; S73. In response to the second determination result, maintain the current scheduling strategy and record the environmental noise characteristics.

[0025] This embodiment describes a feedback closed-loop mechanism based on the judgment result; when the judgment result is the first judgment result, it is confirmed that the current energy efficiency decline is caused by the failure of the scheduling strategy, and the system will further identify the specific reinforcement learning strategy pattern that caused the failure; the identification method can be achieved by matching the specific fault type corresponding to the theoretical residual vector; the system triggers parameter adjustment for the reinforcement learning strategy pattern, such as reducing the exploration rate or starting the retraining process; When the determination result is the second determination result, it is confirmed that the energy efficiency decline is caused by environmental noise. The system will maintain the current scheduling strategy unchanged and record the environmental noise characteristics. The recorded noise characteristics can be used to update the environmental model in the future to prevent the reinforcement learning agent from over-reacting to the noise, i.e., overfitting. By constructing the above-mentioned automated operation and maintenance closed loop, this embodiment clearly distinguishes between the internal causes of strategy failure and the external causes of environmental noise, enabling the system to take differentiated countermeasures, effectively avoiding scheduling strategy oscillations caused by blindly responding to environmental noise, and significantly improving the long-term operational stability of the power computing power collaborative system.

[0026] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A power cooperative scheduling evaluation method based on reinforcement learning, characterized in that, The specific steps include: S1. Collect hardware instruction set parameters and task metadata from the power computing collaborative system, establish a theoretical minimum energy consumption model, and generate an ideal baseline state vector by combining real-time electricity price data and carbon intensity data. S2. Construct a fault feature direction vector based on the scheduling failure mode, and combine it with the interference intensity coefficient to generate a simulated disturbed state vector. S3. Collect real-time running data, calculate the difference between the real-time running data and the ideal baseline state vector, and generate the actual residual vector; S4. Calculate the difference between the simulated disturbed state vector and the ideal reference state vector to generate the theoretical residual vector; S5. Map the real residual vector and the theoretical residual vector to a high-dimensional feature space, calculate the geometric overlap, and generate the coupling coefficient. S6. Compare the coupling coefficient with the preset confidence threshold. If the coupling coefficient is greater than the confidence threshold, it is determined that the energy efficiency decline is caused by the failure of the scheduling strategy, and the first judgment result is output. If the coupling coefficient is less than or equal to the confidence threshold, it is determined that the energy efficiency decline is caused by environmental noise, and the second judgment result is output. S1 includes: S11. By analyzing the hardware instruction set architecture, obtain the physical limit energy consumption data of the chip when executing computationally intensive and memory-intensive operators; S12. Based on physical limit energy consumption data, establish a theoretical minimum energy consumption model that is independent of scheduling strategies and environmental noise; S13. Based on real-time electricity price data and carbon intensity data, use linear programming to calculate the minimum cost and minimum carbon emission trajectory. S14. Combine the theoretical minimum energy consumption model, minimum cost and minimum carbon emission trajectory to generate an ideal baseline state vector; S1 also includes: S15. Perform differential statistics through the instruction flow monitoring window to obtain the instantaneous execution rate of the operator; S16. Calculate the dynamic power consumption of the computing core based on the instantaneous execution rate and the physical limit energy consumption of a single operation of the operator. S17. Obtain the instantaneous data throughput of the task through bandwidth monitoring data; S18. Calculate the transmission power consumption of the subsystem based on the instantaneous data throughput and the energy consumption limit of physical transmission per unit of data. S19. Sum the chip's static power consumption, the calculation core's dynamic power consumption, and the subsystem's transmission power consumption to generate the instantaneous theoretical minimum power, and construct an ideal reference state vector based on the instantaneous theoretical minimum power.

2. The power cooperative scheduling evaluation method based on reinforcement learning according to claim 1, characterized in that, S2 include: S21. Based on historical fault data, extract features of reinforcement learning scheduling failure modes and construct fault feature direction vectors; wherein, the fault feature direction vectors are normalized. S22. Determine the interference intensity coefficient based on the current system load level or historical case fitting; S23. Calculate the product of the fault characteristic direction vector and the interference intensity coefficient to generate the interference operator; S24. Superimpose the disturbance operator onto the ideal reference state vector to generate the simulated disturbed state vector.

3. The power cooperative scheduling evaluation method based on reinforcement learning according to claim 1, characterized in that, S5 include: S51. Using the cosine similarity algorithm, calculate the dot product between the actual residual vector and the theoretical residual vector; S52. Calculate the product of the magnitude of the actual residual vector and the magnitude of the theoretical residual vector; S53. Compare the product with the numerically stable minimum value and select the larger value as the denominator. S54. Divide the dot product by the denominator to generate the coupling coefficient.

4. The power cooperative scheduling evaluation method based on reinforcement learning according to claim 1, characterized in that, It also includes the step of determining the confidence threshold: S01. Obtain the historical labeled dataset and iterate through different candidate thresholds; S02. For each candidate threshold, calculate the harmonic mean of precision and recall to generate an F1 score; S03. Select the candidate threshold that maximizes the F1 score and determine the candidate threshold as the confidence threshold.

5. The power cooperative scheduling evaluation method based on reinforcement learning according to claim 1, characterized in that, It also includes a feedback step based on the judgment result: S71. In response to the first decision result, identify the reinforcement learning strategy pattern that leads to failure; S72, triggers parameter adjustment or retraining processes for reinforcement learning policy patterns; S73. In response to the second determination result, maintain the current scheduling strategy and record the environmental noise characteristics.

Images