A method for collaborative dispatching of intelligent calculation center and carbon based on reinforcement learning

Abstract

The application discloses a kind of based on reinforcement learning's intelligent calculation center calculates electric carbon collaborative scheduling method, it is related to artificial intelligence technical field, including, the source network load storage carbon operation data of intelligent calculation center is collected, and time series calibration is carried out, generates five-dimensional perception dataset;Based on five-dimensional scheduling feature set, source network load storage carbon five-dimensional unified state space is constructed, based on computing power energy storage collaborative weight initialization multi-agent scheduling network, and carry out offline pre-training, form hierarchical distributed decision model;Through uncertainty exploration mechanism and composite reward function, action evaluation is carried out to safe candidate action, generates target collaborative action;Through feedback sample correction multi-agent scheduling network, generate updated scheduling strategy, and carry out market signal adaptation, generate low-carbon operation scheduling scheme.The application generates low-carbon operation scheduling scheme by carrying out market signal adaptation, improves the economy of low-carbon operation scheduling scheme, initiative carbon reduction capacity and market adaptation ability.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for collaborative scheduling of computing power and carbon emissions in intelligent computing centers based on reinforcement learning. Background Technology

[0002] With the rapid development of large-scale AI model training, inference services, and high-performance computing, intelligent computing centers have gradually become important infrastructure for the centralized supply of computing power resources. The operation of intelligent computing centers involves high-density energy consumption of server clusters, network equipment, and cooling equipment, and is also closely related to new energy access, energy storage regulation, electricity price fluctuations, and carbon emission constraints. In order to reduce operating costs and improve the level of low-carbon operation, related technologies have begun to develop from local optimization such as single computing power resource scheduling, power load peak shaving and valley filling, and energy storage charging and discharging control to a comprehensive coordination of computing power resources, power supply, energy storage equipment, cooling load, and carbon emission accounting.

[0003] However, existing technologies still have the following shortcomings: existing solutions are mostly focused on a single objective such as load balancing, electricity cost reduction, or equipment energy consumption optimization, making it difficult to simultaneously characterize the coupling relationship between source-side energy supply, grid-side electricity price, load-side computing power, energy storage status, and carbon emission constraints. In addition, market signals such as carbon trading prices, green certificate prices, and green electricity trading prices are mostly used as ex-post statistics or external reference factors in the current scheduling process, and have not been fully integrated into the online scheduling strategy update process, resulting in insufficient initiative and economy in the low-carbon operation scheduling of intelligent computing centers. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a reinforcement learning-based intelligent computing center computing, electricity, and carbon collaborative scheduling method to address the lack of initiative and economy in the low-carbon operation scheduling of intelligent computing centers.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a method for collaborative scheduling of computing power and carbon emissions in intelligent computing centers based on reinforcement learning, comprising,

[0008] Collect source-grid-load-storage carbon operation data from the intelligent computing center, perform time-series calibration, generate a five-dimensional sensing dataset, and normalize the scheduling features of the five-dimensional sensing dataset to generate a five-dimensional scheduling feature set.

[0009] A unified five-dimensional state space for source, grid, load, storage, and carbon is constructed based on a five-dimensional scheduling feature set, and state mapping is performed to generate a five-dimensional joint state. The computing power and energy storage collaborative scoring of the five-dimensional joint state is performed to generate computing power and energy storage collaborative weights. Based on the computing power and energy storage collaborative weights, a multi-agent scheduling network is initialized and offline pre-training is performed to form a hierarchical distributed decision model.

[0010] The five-dimensional joint state is input into a hierarchical distributed decision model for action masking reasoning, outputting safe candidate actions. The safe candidate actions are evaluated through an uncertainty exploration mechanism and a composite reward function to generate target collaborative actions.

[0011] The system maps the target coordinated actions to instructions, generates coordinated scheduling data, calculates the task performance, power cost, energy storage loss, cooling energy consumption and real-time carbon emission deviation in the coordinated scheduling data, forms feedback samples, corrects the multi-agent scheduling network through feedback samples, generates updated scheduling strategies, adapts to market signals, and generates a low-carbon operation scheduling scheme.

[0012] As a preferred embodiment of the reinforcement learning-based intelligent computing center computation, power, and carbon collaborative scheduling method described in this invention, the specific steps for generating the five-dimensional sensing dataset are as follows:

[0013] Collect source-side energy supply data, grid-side electricity price data, load-side computing power data, energy storage operation data, and carbon constraint data from the intelligent computing center, and then timestamp and organize them to generate five-dimensional raw operation data;

[0014] The sampling frequency of the original five-dimensional operational data is unified and missing time periods are filled in to generate continuous five-dimensional operational data.

[0015] The five-dimensional continuous running data is time-aligned according to the same time index to generate five-dimensional aligned running data, and time offset correction is performed to generate a five-dimensional perception dataset.

[0016] As a preferred embodiment of the reinforcement learning-based intelligent computing center computational power carbon collaborative scheduling method described in this invention, the specific steps for generating the five-dimensional scheduling feature set are as follows:

[0017] The five-dimensional perception dataset is divided into features according to data type to form five-dimensional feature records;

[0018] Unify the dimensions, value ranges, and time indexes in the five-dimensional feature records to generate five-dimensional standard feature records;

[0019] The source-grid-load-storage carbon scheduling index is calculated based on the five-dimensional standard feature record, and feature encoding is performed to generate a five-dimensional scheduling feature set.

[0020] As a preferred embodiment of the reinforcement learning-based intelligent computing center computational power carbon collaborative scheduling method described in this invention, the specific steps for generating the five-dimensional joint state are as follows:

[0021] The state roles of the five-dimensional scheduling feature set are reconstructed to form a five-dimensional state index group;

[0022] Using the load-side state index group in the five-dimensional state index group as the scheduling anchor point, the five-dimensional state index group is modeled with anchor point association to generate cross-dimensional association items, and the source dimension identifier and target dimension identifier are marked to form a cross-dimensional coupling record.

[0023] Using the source dimension identifier and target dimension identifier in the cross-dimensional coupling record as row and column indices, the cross-dimensional coupling record is encoded using an adjacency matrix to form a five-dimensional coupling relationship matrix;

[0024] Five-dimensional state slots are established based on the five-dimensional state index group, and a five-dimensional coupling relationship matrix is ​​embedded to form a unified five-dimensional state space for source, grid, load, and carbon storage.

[0025] The five-dimensional state index group is mapped to the state slots of each dimension in the five-dimensional unified state space of source, grid, load and storage carbon to generate a five-dimensional state carrier.

[0026] The association weights of each state slot in the five-dimensional state carrier are adjusted using the five-dimensional coupling relationship matrix to generate a five-dimensional associated state diagram.

[0027] The five-dimensional relational state diagram is jointly normalized to generate a five-dimensional joint state.

[0028] As a preferred embodiment of the reinforcement learning-based intelligent computing center computing power-carbon collaborative scheduling method described in this invention, the specific steps for generating computing power-energy storage collaborative weights are as follows:

[0029] Extract the load-side computing power state quantity and the energy storage lifetime state quantity from the load-side state slot and the energy storage state slot, and then splice the state features to form a joint state vector.

[0030] By performing task elasticity classification and energy storage lifetime segmentation on the joint state vector, a computing-storage classification profile is obtained.

[0031] The entropy weight TOPSIS evaluation method is used to rank the computing and storage hierarchical profiles based on their closeness, forming a computing and storage synergy order.

[0032] The computing power decision weight and energy storage protection weight are allocated according to the computing power-storage coordination order to generate computing power-storage coordination weight.

[0033] As a preferred embodiment of the reinforcement learning-based intelligent computing center computing power and carbon collaborative scheduling method described in this invention, the specific steps for forming a hierarchical distributed decision model are as follows:

[0034] The state observation encoding of the five-dimensional joint state is performed to form a multi-agent observation vector group;

[0035] Based on the observation vector group of multi-agents, establish local policy channels and global evaluation channels in each dimension, and connect them with state sharing to form a scheduling network to be initialized;

[0036] The initial parameters of the local strategy channels and global evaluation channels in each dimension are adjusted according to the collaborative weight of computing power and energy storage to form an initial multi-agent scheduling network;

[0037] State slicing and reward labeling are performed on the five-dimensional perception dataset to form an offline training sample chain;

[0038] The local policy channels and global evaluation channels of the initial multi-agent scheduling network are trained using offline training sample chains to form a pre-trained scheduling network.

[0039] By using a centralized training and decentralized execution mechanism, the pre-trained scheduling network is divided into state training and state reasoning, forming a hierarchical distributed decision-making model.

[0040] As a preferred embodiment of the reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method described in this invention, the specific steps for outputting safe candidate actions are as follows:

[0041] The five-dimensional joint state is input into the hierarchical distributed decision model to perform action probability inference and generate a candidate action probability table.

[0042] Perform feasibility boundary checks on the action items in the candidate action probability table and generate action disabling flags;

[0043] The candidate action probability table is modified using action disabling flags to generate a safe action probability table. Action items are then filtered from the safe action probability table, and safe candidate actions are output.

[0044] As a preferred embodiment of the reinforcement learning-based intelligent computing center computational power carbon collaborative scheduling method described in this invention, the specific steps for generating the target collaborative action are as follows:

[0045] Continuous state values ​​of new energy output, electricity price, carbon emission factor and computing load are selected from the five-dimensional joint state, and time-series fluctuation analysis is performed to generate state fluctuation sequence.

[0046] Confidence upper bounds are estimated for the state fluctuation sequence to generate uncertainty exploration parameters;

[0047] The exploration ratio of safe candidate actions is adjusted by using uncertainty exploration parameters to obtain exploration candidate actions;

[0048] A composite reward function is used to evaluate the value of candidate actions and generate action evaluation results.

[0049] Candidate actions are selected and explored based on the action evaluation results, and target collaborative actions are generated.

[0050] As a preferred embodiment of the reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method described in this invention, the specific steps for generating collaborative scheduling data are as follows:

[0051] The target coordinated actions are broken down into action types, namely computing power scheduling actions, energy storage regulation actions, cooling regulation actions, and carbon budget actions, and the parameters are expanded to generate coordinated scheduling instructions.

[0052] The scheduling parameters are assigned and the status is synchronized for the coordinated scheduling instructions, generating coordinated scheduling data.

[0053] As a preferred embodiment of the reinforcement learning-based intelligent computing center computational power carbon collaborative scheduling method described in this invention, the specific steps for generating a low-carbon operation scheduling scheme are as follows:

[0054] Collect carbon trading prices, green certificate prices, and green electricity trading prices to form market signal records;

[0055] By revising and updating the carbon budget allocation weight, green electricity consumption weight, and electricity purchase cost weight in the dispatch strategy through market signal recording, a low-carbon operation dispatch plan is generated.

[0056] The beneficial effects of this invention are as follows: By standardizing scheduling features, a five-dimensional scheduling feature set is generated, improving the stability and trainability of multi-objective scheduling decisions; by performing state mapping, a five-dimensional joint state is generated, avoiding the one-sided decision-making of traditional scheduling methods based solely on local states; by allocating computing power decision weights and energy storage protection weights, computing power and energy storage collaborative weights are generated, avoiding the rigidity of policy response caused by using fixed weights in all scenarios, and improving the adaptability of reinforcement learning scheduling strategies under different load and energy storage states; by performing state training and state reasoning partitioning, a hierarchical distributed decision model is formed, which can take into account both the global coordination capability and online reasoning efficiency of scheduling strategies; by using an uncertainty exploration mechanism and a composite reward function to evaluate safe candidate actions and generate target collaborative actions, avoiding task delays caused by simply pursuing electricity cost reduction; and by adapting to market signals, a low-carbon operation scheduling scheme is generated, improving the economy, proactive carbon reduction capability, and market adaptability of the low-carbon operation scheduling scheme. Attached Figure Description

[0057] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart of a reinforcement learning-based intelligent computing center's collaborative scheduling method for electricity and carbon emissions.

[0059] Figure 2 A flowchart for generating a five-dimensional scheduling feature set.

[0060] Figure 3 This is a structural diagram of a hierarchical distributed decision-making model.

[0061] Figure 4 A flowchart for generating a low-carbon operation scheduling scheme. Detailed Implementation

[0062] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0063] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0064] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0065] Reference Figures 1-4 This is one embodiment of the present invention, which provides a method for collaborative scheduling of computing power and carbon emissions in intelligent computing centers based on reinforcement learning, comprising the following steps:

[0066] S1: Collect source-grid-load-storage carbon operation data from the intelligent computing center, perform time-series calibration, generate a five-dimensional sensing dataset, and standardize the scheduling features of the five-dimensional sensing dataset to generate a five-dimensional scheduling feature set.

[0067] S1.1: Collect source-side energy supply data, grid-side electricity price data, load-side computing power data, energy storage operation data, and carbon constraint data from the intelligent computing center, and organize them with timestamps to generate five-dimensional raw operation data; unify the sampling frequency and fill in missing time periods of the five-dimensional raw operation data to generate five-dimensional continuous operation data; align the five-dimensional continuous operation data according to the same time index to generate five-dimensional aligned operation data, and perform time offset correction to generate a five-dimensional sensing dataset.

[0068] The system collects renewable energy output, external power supply, and energy surplus through source-side energy acquisition equipment to form source-side energy supply data. It also collects real-time and time-of-use electricity prices through the grid-side electricity price acquisition interface to form grid-side electricity price data. The system collects computing load, task queue, and task execution time limits through the computing power resource management platform to form load-side computing power data. The system collects state of charge, charging and discharging power, and cycle count through the energy storage controller to form energy storage operation data. Finally, the system collects carbon emission factors, carbon budgets, and carbon emission limits through the carbon emission metering interface to form carbon constraint data. The original acquisition time is retained for all data, including source-side energy supply data, grid-side electricity price data, load-side computing power data, energy storage operation data, and carbon constraint data. The original acquisition time is converted into a unified time format and arranged according to the acquisition object, data type, data value, and acquisition time to form five-dimensional original operation data with a unified time field.

[0069] Using the intelligent computing center scheduling cycle as the unified sampling interval, and the earliest acquisition time in the five-dimensional raw operational data as the first standard sampling time, subsequent standard sampling times are arranged according to the unified sampling interval to form a standard sampling time sequence. For each data record in the five-dimensional raw operational data, the absolute value of the time difference between the acquisition time of the data record and each adjacent standard sampling time is taken, and the standard sampling time with the smallest absolute value of the time difference is taken as the standard sampling time to which the data record belongs. This ensures that the source-side energy supply data, grid-side electricity price data, load-side computing power data, energy storage operation data, and carbon constraint data have a consistent sampling frequency. When there are time positions in the standard sampling time sequence where no data records are configured, the time positions where no data records are configured are taken as missing time periods. For continuous numerical data in the missing time periods, the previous valid data value and the next valid data value in the missing time period are taken as interpolation endpoints, and the supplementary data value is determined according to the time distance ratio between the sampling time of the missing time period and the sampling times of the two interpolation endpoints. For state-type data in the missing time periods, the previous valid state in the missing time period is taken as the supplementary state value to form five-dimensional continuous operational data.

[0070] Data records with the same standard sampling time in the five-dimensional continuous operation data are compiled into the same time index, forming five-dimensional operation records containing source-side energy supply data, grid-side electricity price data, load-side computing power data, energy storage operation data, and carbon constraint data under the same time index, thus forming five-dimensional aligned operation data. The absolute value of the time difference between the original acquisition time and the standard sampling time of each data record is taken as the time offset. The standard sampling time of the time index is used as the corrected sampling time of the data record, and the time offset is written into the offset verification field, so that the five-dimensional operation records under the same time index have a consistent corrected sampling time, generating a five-dimensional sensing dataset.

[0071] S1.2: Divide the five-dimensional sensing dataset into features according to data type to form five-dimensional feature records; unify the dimensions, value ranges and time indexes in the five-dimensional feature records to generate five-dimensional standard feature records.

[0072] According to data type, the operational data in the five-dimensional sensing dataset are categorized into the source-grid-load-storage carbon feature category. Features with the same time index are grouped into a single record, forming a five-dimensional feature record containing feature name, feature category, feature value, and time index. The original physical meaning is preserved according to feature category. Power-related features are standardized to kilowatts, electricity price-related features to yuan per kilowatt-hour, carbon emission-related features to kilograms of carbon dioxide equivalent, and time-related features to a second-level time format. This ensures consistent units of measurement for similar features within the five-dimensional feature record. The minimum feature value under the same feature name within the five-dimensional feature record is used as the lower limit, and the maximum feature value is used as the upper limit. The difference between the current feature value and the lower limit of the value is calculated, and the difference between the upper limit of the value and the lower limit of the value is also calculated. The ratio of the two differences is taken as the normalized feature value. During the normalization process, when the maximum and minimum feature values ​​under the same feature name are the same, the corresponding normalized feature value is uniformly set to zero. When the current feature value exceeds the range of values ​​defined by the minimum and maximum feature values, the normalized feature value is truncated to zero to one, ensuring that the normalized feature values ​​in the five-dimensional standard feature record have a unified and effective range of values. The time index in the five-dimensional feature record is compiled into a continuous sequence number according to the sampling order, and the source-grid-load-storage carbon features under the same continuous sequence number are kept in the same record position to generate the five-dimensional standard feature record.

[0073] S1.3: Calculate the source-grid-load-storage carbon scheduling index based on the five-dimensional standard feature record, and perform feature encoding to generate a five-dimensional scheduling feature set.

[0074] The following parameters are selected from the five-dimensional standard feature records: available new energy power, total energy supply power, current electricity price, previous time indexed electricity price, occupied computing power resources, total available computing power resources, current energy storage power, rated energy storage capacity, electricity consumption carbon emissions, and electricity consumption. The ratio of new energy output to total energy supply power is taken as the source-side green electricity availability. The absolute value of the difference between the current electricity price and the previous time indexed electricity price is taken as the electricity price change. The maximum value between the current electricity price and the previous time indexed electricity price is taken as the electricity price benchmark. When the electricity price benchmark is greater than zero, the ratio of the electricity price change to the electricity price benchmark is taken as the grid-side electricity price volatility. When the electricity price benchmark is equal to zero, the grid-side electricity price volatility is set to zero. The ratio of occupied computing power resources to total available computing power resources is taken as the load-side computing power load rate. The ratio of current energy storage power to rated energy storage capacity is taken as the energy storage state of charge. The ratio of electricity consumption carbon emissions to electricity consumption is taken as the carbon-side carbon emission factor, forming a source-grid-load-storage carbon scheduling index.

[0075] The scheduling indicators of source-grid-load-storage carbon are arranged in the order of source, grid, load, and carbon storage. The availability of green electricity on the source side, the fluctuation of electricity price on the grid side, the computing power load rate on the load side, the energy storage state of charge, and the carbon emission factor on the carbon side under the same time index are written into the same feature vector. The feature vector is bound to the time index and arranged in the order of the time index to generate a five-dimensional scheduling feature set.

[0076] S2: Construct a unified five-dimensional state space for source, grid, load, storage, and carbon based on the five-dimensional scheduling feature set, perform state mapping, generate a five-dimensional joint state, perform a computing power and energy storage collaborative scoring on the five-dimensional joint state, generate computing power and energy storage collaborative weights, initialize a multi-agent scheduling network based on the computing power and energy storage collaborative weights, and perform offline pre-training to form a hierarchical distributed decision model.

[0077] S2.1: Reconstruct the state roles of the five-dimensional scheduling feature set to form a five-dimensional state indicator group; take the load-side state indicator group in the five-dimensional state indicator group as the scheduling anchor point, perform anchor point association modeling on the five-dimensional state indicator group, generate cross-dimensional association items, and mark the source dimension identifier and target dimension identifier to form a cross-dimensional coupling record.

[0078] Features describing energy supply capacity are organized into a source-side state index group, features describing electricity purchase price and supply conditions are organized into a grid-side state index group, features describing computing load and task time limits are organized into a load-side state index group, features describing energy storage availability and charging / discharging capacity are organized into an energy storage state index group, and features describing carbon emission intensity and carbon budget constraints are organized into a carbon-side state index group, forming a five-dimensional state index group. The load-side state index group serves as the scheduling anchor point. The computing load and task time limits in the load-side state index group characterize the current computing demand. The remaining task duration is determined based on the task execution time limit and the corresponding time index. If the task execution time limit is earlier than the corresponding time index, the remaining task duration is set to zero. The normalization range is defined by the minimum and maximum remaining task durations within the same scheduling cycle. The normalized task duration is obtained using a maximum-minimum normalization method. When the normalization range length is zero, the normalized task duration is set to zero. The duration is set to zero; the normalized remaining task duration is converted into task urgency, so that the shorter the remaining task duration, the higher the task urgency; the task urgency and computing load are equally weighted to obtain the current computing power demand intensity. According to the same time index, the current computing power demand intensity is correlated with the green electricity availability in the source-side status index group, the electricity price volatility in the grid-side status index group, the energy storage charge status in the energy storage status index group, and the carbon emission factor in the carbon-side status index group. The product of green electricity availability and current computing power demand intensity is taken as the source-load support correlation value, the average of electricity price volatility and current computing power demand intensity is taken as the grid-load impact correlation value, the product of energy storage charge status and current computing power demand intensity is taken as the storage-load adjustment correlation value, and the average of carbon emission factor and current computing power demand intensity is taken as the carbon load constraint correlation value. The scheduling anchor point, the names of the two related indicators and all the correlation values ​​under the same time index are collected to generate a cross-dimensional correlation item.

[0079] The dimension belonging to the non-load-side state indicator group participating in the cross-dimensional association is taken as the source dimension, and the dimension belonging to the load-side state indicator group is taken as the target dimension, with source dimension identifiers and target dimension identifiers respectively. Data records with time indexes, scheduling anchors, cross-dimensional association items, source dimension identifiers, and target dimension identifiers form cross-dimensional coupled records.

[0080] S2.2: Using the source dimension identifier and target dimension identifier in the cross-dimensional coupling record as row and column indices, the cross-dimensional coupling record is encoded by an adjacency matrix to form a five-dimensional coupling relationship matrix; based on the five-dimensional state index group, a five-dimensional state slot is established and embedded into the five-dimensional coupling relationship matrix to form a unified five-dimensional state space of source, grid, load, storage, and carbon.

[0081] The row and column positions of the adjacency matrix are set according to the fixed dimensional order of source, grid, load, and carbon storage. The source dimension identifier in the cross-dimensional coupling record is matched to the row position of the adjacency matrix, and the target dimension identifier in the cross-dimensional coupling record is matched to the column position of the adjacency matrix. The row and column positions of the cross-dimensional coupling record with cross-dimensional correlation items are configured with correlation markers, and the row and column positions of the cross-dimensional coupling record without cross-dimensional correlation items are configured with empty correlation markers. The correlation markers under the same time index are arranged according to the row and column positions of the adjacency matrix to form a five-dimensional coupling relationship matrix.

[0082] Five-dimensional state slots are established based on the indicator names, indicator values, and time indices in the five-dimensional state indicator group. These slots store state indicators for each dimension under the same time index. Each five-dimensional state slot includes a dimension identifier, state indicator name, state indicator value, and time index. The fixed dimensional order of source-grid-load-storage carbon is used as the arrangement order of the five-dimensional state slots, ensuring that state indicators in the five-dimensional state indicator group are placed in slots with the same time index. The row and column positions of the five-dimensional coupling relationship matrix are bound to the dimension identifiers of the five-dimensional state slots. All correlation values ​​in the cross-dimensional correlation items are written into the row and column positions of the five-dimensional coupling relationship matrix as the correlation strength field between the five-dimensional state slots. Two five-dimensional state slots with numerical correlation strength fields establish a cross-dimensional connection; two five-dimensional state slots with empty correlation strength fields do not establish a cross-dimensional connection, thus forming a unified five-dimensional state space for source-grid-load-storage carbon.

[0083] S2.3: Map the five-dimensional state index group to the state slots of each dimension in the five-dimensional unified state space of source, grid, load, and carbon storage to generate a five-dimensional state carrier; adjust the association weights of each state slot in the five-dimensional state carrier using the five-dimensional coupling relationship matrix to generate a five-dimensional associated state diagram; perform joint normalization on the five-dimensional associated state diagram to generate a five-dimensional joint state.

[0084] State indicators under the same time index are selected from the five-dimensional state indicator group according to the time index. Based on the dimension identifier, indicator name, and indicator value of the state indicator, they are filled into the state slots of each dimension in the unified five-dimensional state space of source-grid-load-storage carbon. Each state slot retains the dimension identifier, indicator name, indicator value, and time index, ensuring that source-side, grid-side, load-side, energy storage, and carbon-side state indicators under the same time index are within the same state structure, generating a five-dimensional state carrier. Using the row and column positions of the five-dimensional coupling relationship matrix as indices, the five-dimensional state slots bound to the row and column positions are read. Each five-dimensional state slot is used as a graph node, and the dimension identifier, state indicator name, state indicator value, and time index within the five-dimensional state slot are used as graph node attributes, forming a five-dimensional state graph node. The row and column positions with numerical content in the association strength field are associated with the five-dimensional state graph node corresponding to the five-dimensional state slot bound to the row position and the column position. Graph edges are formed between the five-dimensional state graph nodes corresponding to the five-dimensional state slots bound to the position. The association strength field is used as the graph edge weight. When the association strength field is empty in the row or column position, a graph edge is formed between the five-dimensional state graph node corresponding to the five-dimensional state slot not bound to the row position and the five-dimensional state graph node corresponding to the five-dimensional state slot bound to the column position. The graph edge weight of the row and column positions is recorded as zero. The five-dimensional state graph nodes, graph edges, and graph edge weights form a five-dimensional association state graph. The state slots in the five-dimensional association state graph are used as graph nodes, and the association weights between the state slots are used as graph edge weights. The state index values ​​in the five-dimensional association state graph are processed using the maximum-minimum normalization method to obtain normalized state values. When the normalization range length under the same state index name is zero, the corresponding normalized state value is set to zero. The maximum association weight under the same time index is used as the association weight normalization benchmark to obtain the normalized association weight. When the maximum association weight is zero, the normalized association weights under the same time index are uniformly set to zero. The normalized state values ​​are arranged in the order of source, grid, load, and carbon storage, and the normalized correlation weights that have cross-dimensional connections with each state slot are arranged after the corresponding normalized state values ​​to generate a five-dimensional joint state.

[0085] S2.4: Extract the computing power state quantity and energy storage lifetime state quantity from the load-side state slot and the energy storage state slot, and splice the state features to form a joint state vector; perform task elasticity classification and energy storage lifetime segmentation on the joint state vector to obtain a computing-storage classification profile.

[0086] The computing power load rate, task queue, and task execution time limit are read from the load-side status slots according to the time index to form the load-side computing power status quantity. The energy storage charge state, charge / discharge power, and cycle count are read from the energy storage status slots under the same time index to form the energy storage lifetime status quantity. The load-side computing power status quantity and the energy storage lifetime status quantity are arranged in the same status record, and the time index, load-side slot identifier, and energy storage slot identifier are retained to form a joint state vector.

[0087] The task elasticity value is determined based on the task execution time limit, the time index corresponding to the moment, and the number of tasks in the task queue. When the number of tasks in the task queue is zero, the corresponding task elasticity value is set as the maximum task elasticity value within the same scheduling period. The task elasticity value range is limited by the minimum and maximum task elasticity values ​​within the same scheduling period, and the task elasticity value range is divided into low elasticity interval, medium elasticity interval, and high elasticity interval, which are respectively marked as the first task elasticity level, the second task elasticity level, and the third task elasticity level to complete the task elasticity classification.

[0088] The intermediate charge value is determined by the midpoint between the lower and upper limits of the allowable state of charge (SOC) recorded in the energy storage operation data. A charge deviation is generated based on the degree to which the SOC deviates from the intermediate charge value. The number of cycles and the charge deviation are processed using a maximum-minimum normalization method to obtain a cycle-use normalized value and a charge deviation normalized value. These two values ​​are then weighted and fused to obtain the energy storage lifetime state value. The minimum and maximum values ​​in the energy storage lifetime state value define the loss range, dividing it into three equal loss intervals: a first loss interval, a second loss interval, and a third loss interval, corresponding to the first, second, and third lifetime segments, respectively. The task elasticity level and the energy storage lifetime segment are written into a joint state vector under the same time index to obtain a computation-storage hierarchical profile.

[0089] S2.5: The entropy weight TOPSIS evaluation method is used to sort the computing and storage hierarchical profiles by proximity, forming a computing and storage collaborative order; computing power decision weight and energy storage protection weight are allocated according to the computing and storage collaborative order to generate computing power and energy storage collaborative weight.

[0090] Task elasticity, computing load rate, energy storage state of charge, and energy storage lifespan status are selected from the computing-storage hierarchical profile as computing-storage collaborative evaluation indicators. Task elasticity and energy storage state of charge are set as positive indicators, while computing load rate and energy storage lifespan status are set as negative indicators. Each computing-storage collaborative evaluation indicator is processed using the maximum-minimum normalization method adapted to the indicator direction to form a computing-storage collaborative evaluation matrix. When the normalization range length of the same computing-storage collaborative evaluation indicator is zero, the corresponding normalized indicator value is set to zero.

[0091] The entropy weight method is used to statistically analyze the information dispersion of each computing-storage collaborative evaluation index under different time indices. The influence of the number of time indices participating in the evaluation on the information entropy value is eliminated by the information entropy normalization coefficient, and the index weights are obtained. When the index proportion is zero, the corresponding information entropy calculation item is set to zero. The computing-storage collaborative evaluation matrix is ​​adjusted using the index weights, and the maximum weighted index value of each computing-storage collaborative evaluation index is determined as the positive ideal value, and the minimum weighted index value is determined as the negative ideal value.

[0092] The distance between each state record and the positive and negative ideal values ​​is measured using Euclidean distance. The proximity is determined based on the proportion of the negative ideal distance in the total distance composed of the positive and negative ideal distances. When the total distance is zero, the proximity is set to 0.5. The state records are arranged from largest to smallest proximity, and the time index, proximity, and sorting order are written into the same sequence record to form a computation-storage collaborative sequence.

[0093] The proximity is directly configured as the computing power decision weight under the same time index, and the complementary value of the proximity is configured as the energy storage protection weight, so that the total weight of the computing power decision weight and the energy storage protection weight are consistent. The computing power decision weight and the energy storage protection weight under the same time index are combined to generate the computing power and energy storage collaborative weight.

[0094] S2.6: Perform state observation encoding on the five-dimensional joint state to form a multi-agent observation vector group; establish local policy channels and global evaluation channels for each dimension based on the multi-agent observation vector group, and perform state sharing connection to form a scheduling network to be initialized.

[0095] The normalized state values ​​and normalized correlation weights in the five-dimensional joint state are read according to the dimensional order of source, grid, load, storage, and carbon. The normalized state values ​​in the same dimension are taken as the observation subjects of this dimension, and the normalized correlation weights that have cross-dimensional connections with this dimension are taken as the correlation observations of this dimension. The observation subjects and correlation observations of this dimension are arranged according to the state index name and the cross-dimensional connection dimension to form single-dimensional observation vectors for each dimension. When the length of a single-dimensional observation vector is less than the preset observation length, zero values ​​are added to the end of the single-dimensional observation vector. When the length of a single-dimensional observation vector exceeds the preset observation length, the vector elements within the preset observation length are retained according to the state index name. The single-dimensional observation vectors of source, grid, load, storage, and carbon are bound to the time index to form a multi-agent observation vector group.

[0096] Each local policy channel includes an observation input layer, a first fully connected layer, a second fully connected layer, and an action probability output layer. The number of nodes in the observation input layer is consistent with the length of the corresponding single-dimensional observation vector. The first fully connected layer is configured with 64 neurons, and the second fully connected layer is configured with 32 neurons. The first and second fully connected layers use a linear rectified activation function. The number of nodes in the action probability output layer is consistent with the number of action items in the corresponding dimension, and the Softmax function is used to output the local action probability.

[0097] The single-dimensional observation vectors of source, grid, load, and carbon storage are concatenated into a global observation vector in a fixed dimensional order, and the local action probabilities of each dimension are concatenated into a joint action probability vector in the same dimensional order. The global evaluation channel includes an evaluation input layer, a first fully connected evaluation layer, a second fully connected evaluation layer, and an evaluation output layer. The evaluation input layer reads the global observation vector and the joint action probability vector. The first fully connected evaluation layer is configured with 128 neurons, and the second fully connected evaluation layer is configured with 64 neurons. The two fully connected evaluation layers use a linear rectified activation function. The evaluation output layer is configured with one neuron and outputs the action evaluation value.

[0098] The local action probabilities output by each dimension's local policy channel are transmitted to the global evaluation channel, and the action evaluation values ​​generated by the global evaluation channel are fed back to each dimension's local policy channel, forming a state-sharing connection; the each dimension's local policy channel, the global evaluation channel, and the state-sharing connection together form the scheduling network to be initialized.

[0099] S2.7: Adjust the initial parameters of the local policy channels and global evaluation channels of each dimension according to the collaborative weight of computing power and energy storage to form an initial multi-agent scheduling network; perform state slicing and reward labeling on the five-dimensional perception dataset to form an offline training sample chain.

[0100] The Xavier initialization method is used to generate baseline policy parameters for each dimension of local policy channels and baseline evaluation parameters for the global evaluation channel. The connection parameters from the second fully connected layer to the action probability output layer in the load-side local policy channel are adjusted using computing power decision weights, and the connection parameters from the second fully connected layer to the action probability output layer in the energy storage local policy channel are adjusted using energy storage protection weights. The baseline policy parameters are maintained for the source-side, network-side, and carbon-side local policy channels. The evaluation input connection parameters corresponding to the load-side and energy storage single-dimensional observation vectors in the global evaluation channel are adjusted using computing power decision weights and energy storage protection weights, respectively. The remaining evaluation input connection parameters are maintained as baseline evaluation parameters. The adjusted local policy parameters and global evaluation parameters form the initial multi-agent scheduling network.

[0101] Using 24 consecutive time indices as state slice windows and one time index as the moving interval between adjacent state slice windows, continuous running records are extracted from the five-dimensional sensing dataset. If the five-dimensional sensing dataset contains less than 24 time indices, all existing time indices are used as a state slice window. The source-side energy supply data, grid-side electricity price data, load-side computing power data, energy storage operation data, and carbon constraint data within each state slice window are arranged into state segments. Adjacent state segments are marked as the current training state and the subsequent training state, respectively, and the last state segment is configured as the training termination state.

[0102] The task performance score, power cost score, energy storage protection score, and carbon constraint score are generated according to the processing method recorded in the original S2.7. The four scores are then fused with equal weights to obtain the reward label value. The current training status, current time index, current joint training action, reward label value, subsequent training status, and training termination mark are written into the same training sample. The training samples are arranged in order of time index to form an offline training sample chain.

[0103] S2.8: Use offline training sample chains to train the local policy channels and global evaluation channels of each dimension in the initial multi-agent scheduling network to form a pre-trained scheduling network; use a centralized training and decentralized execution mechanism to perform state training and state reasoning partitioning on the pre-trained scheduling network to form a hierarchical distributed decision model.

[0104] The current training state, current joint training action, reward label value, subsequent training state, and training termination marker are read in batches from the offline training sample chain according to the time index. Each training batch contains 64 training samples. The local policy channel of each dimension processes the corresponding single-dimensional observation vector in the current training state, outputs the local action probability, and forms the current joint training action from the local action probabilities of each dimension; the target local policy channel processes the subsequent training state to form the subsequent joint training action.

[0105] The current training state and the current joint training action are sent to the global evaluation channel to obtain the current action evaluation value; the subsequent training state and the subsequent joint training action are sent to the target global evaluation channel to obtain the subsequent action evaluation value.

[0106] The target evaluation value is determined according to the following formula:

[0107] ;

[0108] In the formula, This indicates the reward label value. This represents the discount factor and is set to 0.95. This indicates the end of training. This indicates the target's global evaluation channel. Indicates the subsequent training status. This indicates the subsequent combined training movements.

[0109] The global evaluation loss is determined according to the following formula:

[0110] ;

[0111] The local strategy loss is determined by the following formula:

[0112] ;

[0113] In the formula, This indicates the number of training samples included in the training batch. This indicates the global evaluation channel. This represents the joint action policy formed by the local policy channels in each dimension; the Adam optimization method is used to update the global evaluation parameters and the local policy parameters in each dimension, with the learning rate of the global evaluation parameters set to 0.001 and the learning rate of the local policy parameters in each dimension set to 0.0001.

[0114] After each training batch is completed, the current local policy parameters and current global evaluation parameters are synchronized to the target local policy channel and target global evaluation channel using a soft update method, with the parameter synchronization coefficient set to 0.01. A complete traversal of the offline training sample chain constitutes one training epoch; the training epoch is considered complete when the number of training epochs reaches 200, or the rate of change of the global evaluation loss over 10 consecutive training epochs does not exceed [a certain value]. Training is stopped when training ends, and a pre-training scheduling network is formed by the local policy channels and global evaluation channels of each dimension after training ends.

[0115] During training, the global evaluation channel reads the complete source-grid-load-storage carbon state and joint training actions. During inference, the independent processing relationship of each dimension's local policy channel to the corresponding single-dimensional observation vector is retained, and the probabilities of each dimension's local actions are corrected through state sharing connections to form a hierarchical distributed decision model.

[0116] S3: Input the five-dimensional joint state into the hierarchical distributed decision model for action masking reasoning, output safe candidate actions, evaluate the safe candidate actions through an uncertainty exploration mechanism and a composite reward function, and generate target collaborative actions.

[0117] S3.1: Input the five-dimensional joint state into the hierarchical distributed decision model to perform action probability inference and generate a candidate action probability table; perform feasible boundary verification on the action items in the candidate action probability table and generate action disabling markers; use the action disabling markers to correct the candidate action probability table, generate a safe action probability table, and filter action items from the safe action probability table to output safe candidate actions.

[0118] In the hierarchical distributed decision-making model, the state observation layer receives the five-dimensional joint state and forms single-dimensional observation vectors according to the dimensional order of source, grid, load, storage, and carbon. These single-dimensional observation vectors then enter the local policy channels in the local policy layer. Each local policy channel outputs the action probability under the corresponding dimension based on local policy parameters. The action probability includes the action item and the selection probability of the action item. The global evaluation layer receives the action probabilities output by the local policy channels and corrects the selection probability of the action item according to global evaluation parameters, forming a candidate action probability table containing action type, action parameters, and selection probability. The actions in the candidate action probability table... Items are entered into feasible boundary verification according to action type. The computing power scheduling action is compared with the task execution time limit, task migration capacity and computing power load limit. The energy storage regulation action is compared with the upper and lower limits of energy storage state of charge, the upper limit of charging and discharging power and the upper limit of cycle number. The cooling regulation action is compared with the cooling set temperature range and the upper limit of cooling power. The carbon budget action is compared with the remaining carbon budget quota and carbon emission limit. When the action parameters of an action item fall within the corresponding feasible boundary, the action item is configured as available. When the action parameters of an action item exceed the corresponding feasible boundary, the action item is configured as disabled and an action disabled flag is generated.

[0119] The candidate action probability table and action disabling flags are matched item by item. Actions marked as disabled have their selection probability set to zero, while actions marked as enabled retain their selection probability. The action items with retained selection probabilities are rearranged, and their selection probabilities are updated according to the ratio of the rearranged selection probability to the sum of all retained selection probabilities, thus generating a safe action probability table.

[0120] A safety screening threshold is set based on the selection probability distribution in the safety action probability table. After re-normalization, the selection probability of each action item in the safety action probability table is within the range of zero to one. The average selection probability of all retained action items is taken as the safety screening threshold, with a value range of 0-1. The range of the safety screening threshold is set based on the principle of action probability normalization. Using the average selection probability of all retained action items as the safety screening threshold can adjust the screening scale according to changes in the number of action items and the concentration of action probabilities, avoiding too many or too few safety candidate actions due to a fixed threshold. The selection probability of each action item in the safety action probability table is compared with the safety screening threshold. Action items with a selection probability greater than or equal to the safety screening threshold are retained as candidate action items, and action items with a selection probability less than the safety screening threshold are removed. The retained candidate action items are arranged in descending order of selection probability. The candidate action item with the highest selection probability is selected as the primary action item of the safety candidate action, and the candidate action item with the second highest selection probability is selected as the alternative action item of the safety candidate action. The primary action item and the alternative action item together form the safety candidate action.

[0121] S3.2: Select continuous state values ​​of new energy output, electricity price, carbon emission factor and computing load from the five-dimensional joint state, and perform time series fluctuation analysis to generate state fluctuation sequence; estimate the confidence upper bound of the state fluctuation sequence to generate uncertainty exploration parameters.

[0122] Continuous state values ​​of renewable energy output, electricity price, carbon emission factor, and computing load are extracted from the five-dimensional joint state according to the time index, while maintaining the arrangement relationship of renewable energy output, electricity price, carbon emission factor, and computing load under the same time index. Each type of continuous state value is arranged in chronological order according to the time index, forming a renewable energy output state chain, an electricity price state chain, a carbon emission factor state chain, and a computing load state chain. The absolute value of the difference between the continuous state value under the current time index and the continuous state value under the previous time index is taken as the adjacent fluctuation quantity. The ratio of the adjacent fluctuation quantity to the continuous state value under the previous time index is taken as the adjacent volatility rate. When the continuous state value under the previous time index is zero, the adjacent fluctuation quantity is taken as the adjacent volatility rate. The adjacent fluctuation quantity and adjacent volatility rate of renewable energy output, electricity price, carbon emission factor, and computing load are recorded according to the time index to generate a state fluctuation sequence.

[0123] Using 12 consecutive time indices as the current time window, the adjacent volatility corresponding to new energy output, electricity price, carbon emission factor and computing load is read from the state volatility sequence. If the state volatility sequence contains less than 12 time indices, all existing adjacent volatility is used as the current time window. The average value and number of observations of adjacent volatility of various continuous state values ​​within the current time window are calculated, and the total number of corresponding adjacent volatility formed up to the current time index is calculated.

[0124] The confidence upper bounds for various continuous state values ​​are determined according to the following formula:

[0125] ;

[0126] In the formula, Indicates the first The upper confidence bound for continuous state values. This represents the average volatility of adjacent values ​​within the current time window. Indicates the total number of observations. This indicates the number of observations within the current time window; when the total number of observations is 1, the confidence extension is set to zero.

[0127] Calculate the confidence upper bounds for new energy output, electricity price, carbon emission factor, and computing load separately, and use the maximum-minimum normalization method to form the exploration intensity value for each dimension. When the four confidence upper bounds are consistent, set the exploration intensity value for each dimension to zero. Arrange the confidence upper bounds and corresponding exploration intensity values ​​in the order of new energy output, electricity price, carbon emission factor, and computing load to form the uncertainty exploration parameters.

[0128] S3.3: Adjust the exploration ratio of safe candidate actions using uncertainty exploration parameters to obtain exploration candidate actions; use a composite reward function to evaluate the action value of exploration candidate actions and generate action evaluation results; select exploration candidate actions according to action evaluation results and generate target collaborative actions.

[0129] Read the exploration intensity values ​​of each dimension in the uncertainty exploration parameters, and determine the exploration ratio based on the mean of the exploration intensity values ​​of each dimension; reallocate the original selection probability and uniform exploration probability in the safe candidate actions according to the exploration ratio, so that when the uncertainty is the highest, the chance of each safe candidate action being explored is increased, and when the uncertainty is the lowest, the original selection probability is maintained as the dominant role; arrange the safe candidate actions according to the reallocated exploration action probabilities to obtain the exploration candidate actions.

[0130] The action parameters of the candidate actions are applied to the current state value in the five-dimensional joint state to form the action evaluation state after one scheduling cycle. The remaining task duration and the number of task queues are updated based on the amount of computing resources allocated, the amount of task migration, and the task delay duration. The energy storage charge state is updated based on the charging and discharging power and the duration, and the energy storage lifetime state value is updated based on the fluctuation of the charging and discharging power. The cooling energy consumption value is determined based on the cooling power and the duration. The real-time carbon emission is determined based on the computing power consumption, energy storage consumption, cooling consumption, and carbon emission factor. The updated remaining task duration, power cost value, energy storage lifetime state value, cooling energy consumption value, and real-time carbon emission are used to form the action evaluation record.

[0131] The following incentive items were determined from the action evaluation records: task performance incentive, electricity cost incentive, energy storage protection incentive, cooling energy consumption incentive, and carbon emission control incentive. The task performance incentive reflects the degree to which the task was completed within the task execution time limit; the electricity cost incentive reflects the electricity cost level corresponding to the candidate action; the energy storage protection incentive reflects the impact of the candidate action on the energy storage life status; the cooling energy consumption incentive reflects the energy consumption level corresponding to the cooling regulation action; and the carbon emission control incentive reflects the degree to which real-time carbon emissions occupy the carbon budget. All incentive items were normalized to zero to one using the maximum-minimum normalization method.

[0132] The composite reward value is determined according to the following formula:

[0133] ;

[0134] In the formula, This indicates the task completion reward item. This indicates an electricity cost incentive item. This indicates the energy storage protection reward items. This indicates a cooling energy consumption bonus item. This represents the carbon emission control reward item; the task performance weight is 0.30, used to prioritize ensuring that computing tasks are completed within the task execution time limit; the electricity cost weight is 0.20, used to limit the electricity cost corresponding to the scheduling action; the energy storage protection weight is 0.20, used to suppress the impact of frequent charging and discharging on the energy storage life status; the cooling energy consumption weight is 0.10, used to control the auxiliary energy consumption generated by cooling operation; and the carbon emission control weight is 0.20, used to constrain the real-time carbon emission amount's occupation of the carbon budget. Each weight is configured according to the target level of prioritizing task performance, balancing electricity cost, energy storage protection and carbon emission control, and constraining cooling energy consumption in the intelligent computing center scheduling, and the total weight is 1. The composite reward value is used to adjust the probability of exploration actions to form an action evaluation value, and the exploration candidate action, exploration action probability, composite reward value and action evaluation value are written into the same record to generate the action evaluation result.

[0135] The action evaluation results are sorted from largest to smallest according to the action evaluation value. The exploration candidate action with the largest action evaluation value is determined as the target cooperative action. When the action evaluation values ​​are the same, the exploration candidate action with the largest composite reward value is retained. When the composite reward values ​​are still the same, the exploration candidate action with the highest probability of exploration action is retained.

[0136] S4: Map instructions to the target coordinated actions, generate coordinated scheduling data, calculate task performance, power cost, energy storage loss, cooling energy consumption and real-time carbon emission deviation in the coordinated scheduling data, form feedback samples, correct the multi-agent scheduling network through feedback samples, generate updated scheduling strategies, adapt to market signals, and generate low-carbon operation scheduling schemes.

[0137] S4.1: Decompose the target collaborative actions into action types, forming computing power scheduling actions, energy storage regulation actions, cooling regulation actions, and carbon budget actions, and expand the parameters to generate collaborative scheduling instructions; perform scheduling parameter placement and status synchronization on the collaborative scheduling instructions to generate collaborative scheduling data.

[0138] Based on the action type, action records involving task migration, task postponement, and computing resource allocation are classified as computing power scheduling actions; action records involving charging, discharging, charging / discharging power adjustment, and state of charge maintenance are classified as energy storage control actions; action records involving cooling set temperature, cooling power, and cooling operation level adjustment are classified as cooling regulation actions; and action records involving carbon budget reservation, carbon budget allocation, and carbon emission credit deduction are classified as carbon budget actions. Parameter expansion completes the execution object, execution parameters, and execution time index according to the action type. Computing power scheduling actions are expanded to include task identifier, computing resource allocation amount, task migration direction, and task execution time; energy storage control actions are expanded to include energy storage device identifier, charging / discharging direction, charging / discharging power, and duration; cooling regulation actions are expanded to include cooling device identifier, cooling set temperature, cooling power, and operation level; and carbon budget actions are expanded to include carbon budget object, budget allocation amount, budget deduction amount, and budget effective time. This yields a coordinated scheduling instruction that includes computing power scheduling instructions, energy storage control instructions, cooling regulation instructions, and carbon budget instructions.

[0139] The scheduling parameters are assigned according to the execution object and execution time index in the collaborative scheduling instructions. The computing power scheduling instructions are assigned to the computing power task execution record, the energy storage regulation instructions are assigned to the energy storage operation record, the cooling regulation instructions are assigned to the cooling operation record, and the carbon budget instructions are assigned to the carbon budget record. The computing power task execution status, energy storage charging and discharging status, cooling setting status, and carbon budget allocation status are recorded according to the same execution time index, so that the execution parameters of the collaborative scheduling instructions are converted into a calculable operation status record, generating collaborative scheduling data.

[0140] S4.2: Calculate task performance, power cost, energy storage loss, cooling energy consumption and real-time carbon emission deviation in the collaborative scheduling data to form feedback samples; correct the multi-agent scheduling network through feedback samples and generate updated scheduling strategies.

[0141] Based on the execution time index in the collaborative scheduling data, the actual completion time of the computing task, actual power consumption, actual energy storage cycle count, actual charging and discharging power, actual cooling energy consumption, and actual real-time carbon emissions are read respectively. The actual completion time of the computing task is verified against the task execution time limit to obtain the task performance deviation. The actual power cost is compared with the expected power cost value corresponding to the target collaborative action to obtain the power cost deviation. The actual energy storage loss value is compared with the expected energy storage loss value to obtain the energy storage loss deviation. The actual cooling energy consumption is compared with the expected cooling energy consumption value to obtain the cooling energy consumption deviation. The actual real-time carbon emissions are compared with the expected real-time carbon emissions to obtain the real-time carbon emissions deviation.

[0142] The maximum-minimum normalization method is used to process task performance deviation, power cost deviation, energy storage loss deviation, cooling energy consumption deviation, and real-time carbon emission deviation, and the normalized deviation is converted into corresponding feedback values. The task performance feedback value, power cost feedback value, energy storage protection feedback value, cooling energy consumption feedback value, and carbon emission control feedback value are processed according to the reward weight configuration in S3.3 to obtain the feedback reward value. The five-dimensional joint state, target coordinated action, feedback reward value, five-dimensional joint state after coordinated scheduling data execution, and execution termination flag are written into the same record to form a feedback sample.

[0143] Feedback samples are added to the offline training sample chain, and the global evaluation parameters and local policy parameters of each dimension are updated using the global evaluation loss and local policy loss in S2.8. Feedback sample training uses a global evaluation learning rate of 0.0001 and a local policy learning rate of 0.00001, with parameter updates completed every 32 accumulated feedback samples. The updated local policy channels and global evaluation channels form an updated multi-agent scheduling network. The action probability distribution, local policy parameters, and global evaluation parameters output by the updated multi-agent scheduling network are combined to generate an updated scheduling policy.

[0144] S4.3: Collect carbon trading prices, green certificate prices, and green electricity trading prices to form market signal records; use market signal records to correct and update the carbon budget allocation weight, green electricity consumption weight, and electricity purchase cost weight in the dispatch strategy to generate a low-carbon operation dispatch plan.

[0145] Carbon trading prices are collected from data released in the carbon trading market, green certificate prices from data released in the green certificate trading market, and green electricity trading prices from data released in the green electricity trading market. The release time, price value, and price effective period are retained. The price effective period is matched with the scheduling time index in the updated scheduling strategy, ensuring that carbon trading prices, green certificate prices, and green electricity trading prices exist under the same scheduling time index, thus forming a market signal record.

[0146] The upper limits of carbon trading prices, green certificate prices, and green electricity trading prices are determined from the market signal records corresponding to the most recent 30 price effective periods. The carbon cost pressure value, green certificate incentive strength value, and green electricity purchase cost value are formed by the ratio of each current price to the corresponding upper limit of price. When the corresponding upper limit of price is zero, the corresponding market signal strength is set to zero.

[0147] The original carbon budget allocation weight, original green electricity consumption weight, and original electricity purchase cost weight are retrieved from the updated dispatch strategy. The relative proportion of the carbon budget allocation weight is increased by using the carbon cost pressure value, the relative proportion of the green electricity consumption weight is increased by using the green certificate incentive intensity value, and the relative proportion of the electricity purchase cost weight is increased by using the green electricity purchase cost value. The three weights after the correction are normalized to keep the total weight of the carbon budget allocation weight, green electricity consumption weight, and electricity purchase cost weight consistent. When all three corrected weights are zero, the original carbon budget allocation weight, original green electricity consumption weight, and original electricity purchase cost weight are retained.

[0148] The revised carbon budget allocation weight, green electricity consumption weight, and electricity purchase cost weight are written into the updated scheduling strategy. The updated scheduling strategy is then applied to the execution order of computing tasks, the energy storage charging and discharging schedule, the cooling operation settings, and the carbon budget allocation process. This enables computing scheduling, electricity use, energy storage regulation, and carbon emission control to be executed collaboratively under the same scheduling time index, generating a low-carbon operation scheduling scheme.

[0149] This embodiment also provides a computer device applicable to the reinforcement learning-based intelligent computing center computing, power, and carbon collaborative scheduling method, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the reinforcement learning-based intelligent computing center computing, power, and carbon collaborative scheduling method as proposed in the above embodiment.

[0150] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0151] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the reinforcement learning-based intelligent computing center computer-computer collaborative scheduling method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0152] In summary, this invention improves the stability and trainability of multi-objective scheduling decisions by standardizing scheduling features to generate a five-dimensional scheduling feature set. It generates a five-dimensional joint state through state mapping, avoiding the biased decisions made by traditional scheduling methods based solely on local states. By allocating computing power decision weights and energy storage protection weights, it generates computing power and energy storage collaborative weights, preventing the rigidity of policy responses caused by using fixed weights in all scenarios and improving the adaptability of reinforcement learning scheduling strategies under different load and energy storage states. Through state training and state reasoning partitioning, it forms a hierarchical distributed decision model that balances the global coordination capability and online reasoning efficiency of scheduling strategies. It evaluates safe candidate actions through an uncertainty exploration mechanism and a composite reward function, generating target collaborative actions to avoid task delays caused by simply pursuing electricity cost reduction. Finally, by adapting to market signals, it generates a low-carbon operation scheduling scheme, improving the economy, proactive carbon reduction capability, and market adaptability of the low-carbon operation scheduling scheme.

[0153] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for collaborative scheduling of computing power and carbon emissions in intelligent computing centers based on reinforcement learning, characterized in that: include, Collect source-grid-load-storage carbon operation data from the intelligent computing center, perform time-series calibration, generate a five-dimensional sensing dataset, and normalize the scheduling features of the five-dimensional sensing dataset to generate a five-dimensional scheduling feature set. A unified five-dimensional state space for source, grid, load, storage, and carbon is constructed based on a five-dimensional scheduling feature set, and state mapping is performed to generate a five-dimensional joint state. The computing power and energy storage collaborative scoring of the five-dimensional joint state is performed to generate computing power and energy storage collaborative weights. Based on the computing power and energy storage collaborative weights, a multi-agent scheduling network is initialized and offline pre-training is performed to form a hierarchical distributed decision model. The five-dimensional joint state is input into a hierarchical distributed decision model for action masking reasoning, outputting safe candidate actions. The safe candidate actions are evaluated through an uncertainty exploration mechanism and a composite reward function to generate target collaborative actions. The system maps the target coordinated actions to instructions, generates coordinated scheduling data, calculates the task performance, power cost, energy storage loss, cooling energy consumption and real-time carbon emission deviation in the coordinated scheduling data, forms feedback samples, corrects the multi-agent scheduling network through feedback samples, generates updated scheduling strategies, adapts to market signals, and generates a low-carbon operation scheduling scheme.

2. The reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method as described in claim 1, characterized in that: The specific steps for generating the five-dimensional perception dataset are as follows: Collect source-side energy supply data, grid-side electricity price data, load-side computing power data, energy storage operation data, and carbon constraint data from the intelligent computing center, and then timestamp and organize them to generate five-dimensional raw operation data; The sampling frequency of the original five-dimensional operational data is unified and missing time periods are filled in to generate continuous five-dimensional operational data. The five-dimensional continuous running data is time-aligned according to the same time index to generate five-dimensional aligned running data, and time offset correction is performed to generate a five-dimensional perception dataset.

3. The reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method as described in claim 2, characterized in that: The specific steps for generating the five-dimensional scheduling feature set are as follows: The five-dimensional perception dataset is divided into features according to data type to form five-dimensional feature records; Unify the dimensions, value ranges, and time indexes in the five-dimensional feature records to generate five-dimensional standard feature records; The source-grid-load-storage carbon scheduling index is calculated based on the five-dimensional standard feature record, and feature encoding is performed to generate a five-dimensional scheduling feature set.

4. The reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method as described in claim 3, characterized in that: The specific steps for generating the five-dimensional joint state are as follows: The state roles of the five-dimensional scheduling feature set are reconstructed to form a five-dimensional state index group; Using the load-side state index group in the five-dimensional state index group as the scheduling anchor point, the five-dimensional state index group is modeled with anchor point association to generate cross-dimensional association items, and the source dimension identifier and target dimension identifier are marked to form a cross-dimensional coupling record. Using the source dimension identifier and target dimension identifier in the cross-dimensional coupling record as row and column indices, the cross-dimensional coupling record is encoded using an adjacency matrix to form a five-dimensional coupling relationship matrix; Five-dimensional state slots are established based on the five-dimensional state index group, and a five-dimensional coupling relationship matrix is ​​embedded to form a unified five-dimensional state space for source, grid, load, and carbon storage. The five-dimensional state index group is mapped to the state slots of each dimension in the five-dimensional unified state space of source, grid, load and storage carbon to generate a five-dimensional state carrier. The association weights of each state slot in the five-dimensional state carrier are adjusted using the five-dimensional coupling relationship matrix to generate a five-dimensional associated state diagram. The five-dimensional relational state diagram is jointly normalized to generate a five-dimensional joint state.

5. The reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method as described in claim 4, characterized in that: The specific steps for generating the computing power and energy storage collaborative weights are as follows: Extract the load-side computing power state quantity and the energy storage lifetime state quantity from the load-side state slot and the energy storage state slot, and then splice the state features to form a joint state vector. By performing task elasticity classification and energy storage lifetime segmentation on the joint state vector, a computing-storage classification profile is obtained. The entropy weight TOPSIS evaluation method is used to rank the computing and storage hierarchical profiles based on their closeness, forming a computing and storage synergy order. The computing power decision weight and energy storage protection weight are allocated according to the computing power-storage coordination order to generate computing power-storage coordination weight.

6. The reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method as described in claim 5, characterized in that: The specific steps for forming a hierarchical distributed decision-making model are as follows. The state observation encoding of the five-dimensional joint state is performed to form a multi-agent observation vector group; Based on the observation vector group of multi-agents, establish local policy channels and global evaluation channels in each dimension, and connect them with state sharing to form a scheduling network to be initialized; The initial parameters of the local strategy channels and global evaluation channels in each dimension are adjusted according to the collaborative weight of computing power and energy storage to form an initial multi-agent scheduling network; State slicing and reward labeling are performed on the five-dimensional perception dataset to form an offline training sample chain; The local policy channels and global evaluation channels of the initial multi-agent scheduling network are trained using offline training sample chains to form a pre-trained scheduling network. By using a centralized training and decentralized execution mechanism, the pre-trained scheduling network is divided into state training and state reasoning, forming a hierarchical distributed decision-making model.

7. The reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method as described in claim 6, characterized in that: The specific steps for outputting the safe candidate action are as follows: The five-dimensional joint state is input into the hierarchical distributed decision model to perform action probability inference and generate a candidate action probability table. Perform feasibility boundary checks on the action items in the candidate action probability table and generate action disabling flags; The candidate action probability table is modified using action disabling flags to generate a safe action probability table. Action items are then filtered from the safe action probability table, and safe candidate actions are output.

8. The reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method as described in claim 7, characterized in that: The specific steps for generating the target cooperative action are as follows. Continuous state values ​​of new energy output, electricity price, carbon emission factor and computing load are selected from the five-dimensional joint state, and time-series fluctuation analysis is performed to generate state fluctuation sequence. Confidence upper bounds are estimated for the state fluctuation sequence to generate uncertainty exploration parameters; The exploration ratio of safe candidate actions is adjusted by using uncertainty exploration parameters to obtain exploration candidate actions; A composite reward function is used to evaluate the value of candidate actions and generate action evaluation results. Candidate actions are selected and explored based on the action evaluation results, and target collaborative actions are generated.

9. The reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method as described in claim 8, characterized in that: The specific steps for generating the collaborative scheduling data are as follows: The target collaborative actions are broken down into action types, namely computing power scheduling actions, energy storage regulation actions, cooling regulation actions, and carbon budget actions, and the parameters are expanded to generate collaborative scheduling instructions. The scheduling parameters are assigned and the status is synchronized for the coordinated scheduling instructions, generating coordinated scheduling data.

10. The reinforcement learning-based intelligent computing center computing power carbon collaborative scheduling method as described in claim 9, characterized in that: The specific steps for generating a low-carbon operation scheduling scheme are as follows. Collect carbon trading prices, green certificate prices, and green electricity trading prices to form market signal records; By revising and updating the carbon budget allocation weight, green electricity consumption weight, and electricity purchase cost weight in the dispatch strategy through market signal recording, a low-carbon operation dispatch plan is generated.

Images