An intelligent scheduling strategy simulation evaluation method and system based on multi-dimensional time sequence data playback

Abstract

The application relates to the technical field of reinforcement learning, and particularly discloses an intelligent scheduling strategy simulation evaluation method and system based on multi-dimensional time sequence data playback, which comprises the following steps: collecting heterogeneous device indexes, container events and historical scheduling logs, performing time alignment and structured processing to form a main event sequence containing event identifiers and decision trigger identifiers, establishing a historical strategy branch and a to-be-evaluated strategy branch receiving the same events, determining an environment state at a decision trigger position and calling a reinforcement learning scheduling strategy with fixed parameters, updating corresponding branches according to historical scheduling actions and to-be-evaluated scheduling actions, associating execution results and evaluation indexes with event identifiers and pairing comparison, and generating a strategy evaluation result. Through the same event input, branch state isolation and event-level pairing, different strategies can independently evolve under consistent resource requests and environment conditions, and the evaluation result can be traced back according to the resource request event.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning technology, and in particular to a simulation evaluation method and system for intelligent scheduling strategies based on multi-dimensional time-series data playback. Background Technology

[0002] In the field of reinforcement learning technology, data acquisition, state representation, policy calculation, action execution, feedback recording, and model updating are typically carried out around intelligent agents, environmental states, action commands, and reward signals. The training program reads sensor records, business logs, and equipment operating parameters, inputs the state variables into the neural network, selects scheduling actions based on the network output, and adjusts the network parameters according to the state changes and reward values ​​returned by the environment. Traditional intelligent scheduling strategy simulation evaluation methods and systems involve cleaning, aligning, and replaying historical multi-dimensional time-series data by timestamp for task allocation, resource allocation, or equipment operation sequence. Load, queues, locations, energy consumption, and equipment states at each moment are input into the simulation environment. Reinforcement learning strategies generate scheduling commands, the simulation program drives the corresponding objects to execute them, and records rewards, completion times, resource usage, and action sequences. Finally, the evaluation results of different strategies are statistically analyzed and compared based on preset indicators.

[0003] Traditional intelligent scheduling strategy evaluation relies on offline processing of heterogeneous hardware metrics, container events, and scheduling logs. The time base format and recording granularity of each data source differ. During event replay, there is a lack of stable correspondence between resource requests, environmental states, and historical decisions. It is difficult to maintain consistency in the state evolution conditions formed by the independent operation of different strategies. Furthermore, it is difficult to trace the aggregated metrics back to the same resource request event. As a result, strategy comparisons are affected by differences in input and state evolution. Changes in reinforcement learning strategy versions can also lead to inconsistencies in the decision-making basis within the same evaluation process. Summary of the Invention

[0004] This invention aims to provide a simulation evaluation method and system for intelligent scheduling strategies based on multi-dimensional time-series data playback. By performing unified time alignment and structured processing on the operating indicators of heterogeneous computing devices, container platform events, and historical scheduling logs, the historical strategy branch states and the branch states of the strategy to be evaluated are constructed under the same baseline state and the same external event input conditions. When a resource request event triggers a scheduling decision, the event progression is frozen, so that the reinforcement learning scheduling strategy to be evaluated determines the scheduling action only based on events that have occurred before the decision time. This enables paired simulation evaluation of the historical scheduling strategy and the scheduling strategy to be evaluated under consistent time-series conditions, state conditions, and event conditions.

[0005] To achieve the above objectives, this invention provides a simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback, comprising the following steps: The system collects operational metrics from heterogeneous computing devices, container platform events, and historical scheduling logs. After time alignment and structured processing, it generates a main event sequence containing event identifiers and decision trigger identifiers. Based on this main event sequence, it establishes historical policy branch states and policy branch states to be evaluated, and inputs the same external event into both. When a resource request event with the decision trigger identifier is read, the event progression is paused. The environment state is determined based on the processed events, and the resource request event and the environment state are input with a fixed-parameter version of the reinforcement learning scheduling policy to be evaluated to obtain the scheduling action to be evaluated. When no resource request event with the decision trigger identifier is read, the historical policy branch states and policy branch states to be evaluated continue to advance. The historical policy branch states are updated based on the historical scheduling actions in the historical scheduling logs, and the policy branch states to be evaluated are updated based on the scheduling action to be evaluated. The event identifier is used to associate the action execution results and evaluation metrics of the two policy branch states. The action execution results and evaluation metrics corresponding to the same event identifier are paired and compared to generate a policy evaluation result.

[0006] Furthermore, the generation of the main event sequence includes: acquiring the original records of the heterogeneous computing device operation indicators, container platform events, and historical scheduling logs; extracting the collection time, occurrence time, data source identifier, and original record sequence number; converting each time expression to a unified time base to generate a unified timestamp record; stably sorting the unified timestamp record according to the occurrence time, data source priority, and original record sequence number; when multiple occurrence times are in the same alignment window, maintaining a deterministic order according to the data source priority to generate an aligned event record; converting the aligned event record into a unified field, and writing the event identifier and the decision trigger identifier according to the resource request type and the historical scheduling action association relationship, respectively.

[0007] Furthermore, the establishment of the historical strategy branch state and the strategy branch state to be evaluated includes: obtaining the device resource topology, available node resources, container lifecycle state, and scheduling request queue before the occurrence of the first external event in the main event sequence; removing state records with timestamps later than the occurrence time of the first external event; generating a baseline state snapshot; copying the baseline state snapshot to a first state namespace and a second state namespace that are isolated from each other; writing the historical strategy branch identifier and the strategy branch to be evaluated identifier into them respectively; and freezing the version number of the baseline state snapshot; positioning the event cursors of both strategy branch states before the first external event, so that the two strategy branches have a consistent event advancement starting point.

[0008] The baseline state snapshot is preferably generated in the following manner: Obtain the device resource topology, total node resources, allocated node resources, container lifecycle status, and pending scheduling request queue recorded last before the occurrence of the first external event; associate state records from different data sources according to resource object identifiers; delete state records whose occurrence time is later than the occurrence time of the first external event; generate candidate state records; compare the total node resources with the allocated node resources; retain the corresponding candidate state record when the allocated node resources do not exceed the total node resources and the container lifecycle status and resource occupancy relationship are consistent; replace the previous state record whose occurrence time is closest to and earlier than the first external event; solidify the valid state record according to the fixed field order of device resource topology, available node resources, container lifecycle status, and pending scheduling request queue, and write the state version number and snapshot generation time.

[0009] Further, when a resource request event with the aforementioned decision trigger identifier is read, the decision trigger identifier value, request lifecycle state, and event identifier of the current resource request event are read. When the decision trigger identifier value is valid and the request lifecycle state is in a pending scheduling state, the current event cursors of the two policy branch states are recorded, reading external events that occurred later than the current resource request event is prohibited, and a pause boundary record is generated. The available resources of the node, container resource occupancy, pending scheduling request queue, and device connection state of the policy branch state to be evaluated before the pause boundary record are obtained, and state changes caused by the current resource request event and events that occurred later than the pause boundary record are excluded, generating the environment state. The current resource request event and the environment state are structured and encoded according to a fixed input field order, and the encoded result is input into the reinforcement learning scheduling policy to be evaluated with a fixed parameter version. The resource allocation object and scheduling execution type output by the policy are received and parsed to generate the scheduling action to be evaluated.

[0010] The generation of the environment state includes: obtaining computing resource requirements, storage resource requirements, device type constraints, and container deployment constraints from the current resource request event; obtaining the online status, available resources, and container carrying status of all heterogeneous computing devices from the branch status of the strategy to be evaluated, forming request constraint data and device status data; filtering heterogeneous computing devices with valid online status, device types that meet the device type constraints, and available resources that are not lower than the computing resource requirements and storage resource requirements from the device status data; excluding heterogeneous computing devices with deployment conflicts based on the container deployment constraints, generating candidate device records; arranging the candidate device records in a fixed order according to device identifiers; determining the remaining resources, number of running containers, number of request queues, and device connection status of each candidate device; and associating the determination results with the request constraint data.

[0011] Furthermore, the updating of the two policy branch states includes: parsing the resource request identifier, target device identifier, resource allocation amount, and scheduling execution type in the historical scheduling action and the scheduling action to be evaluated, respectively; associating the corresponding resource request event based on the resource request identifier to generate historical action records and action records to be evaluated; matching the historical action records and action records to be evaluated with the device online status, node available resources, and request lifecycle status in the corresponding policy branch states to determine resource deduction, container state migration, and request queue removal items, generating a first state change record and a second state change record; writing the first state change record into the historical policy branch state, and writing the second state change record into the policy branch state to be evaluated.

[0012] The historical scheduling actions are determined based on the event identifier, resource request identifier, and occurrence time of the current resource request event. Candidate historical actions with the same resource request identifier and an action recording time no earlier than the request occurrence time are selected from the historical scheduling logs. If multiple candidate historical actions exist, the candidate historical action that the scheduler has confirmed execution of and has the earliest action recording time is selected first. If no candidate historical action has been confirmed execution by the scheduler, the candidate historical action with the earliest action recording time and which has not been revoked is selected. The target device identifier, resource allocation amount, and scheduling execution type in the selected historical action are determined as the historical scheduling action.

[0013] Furthermore, the generation of the strategy evaluation results includes: based on the event identifier of each resource request event, obtaining the action execution status, resource request completion status, and evaluation index value from the two strategy branch states respectively; establishing a correspondence between the two sets of obtained results belonging to the same event identifier to generate an event pairing record; comparing the historical index value and the index value to be evaluated for the same evaluation index in the event pairing record, and combining the action execution status and resource request completion status of each of the two strategy branch states to determine the single event comparison result for each resource request event; summarizing multiple single event comparison results according to the evaluation index category, and counting the number of events in which the strategy to be evaluated is better than, equal to, or worse than the historical strategy.

[0014] The present invention also provides an intelligent scheduling strategy simulation and evaluation system based on multi-dimensional time-series data playback to support the above-mentioned method, including a data processing module, a state storage module, a playback execution module, a strategy adaptation interface, and an evaluation module. The data processing module receives operational metrics, container platform events, and historical scheduling logs through the heterogeneous computing device monitoring interface, container platform interface, and scheduler log interface. After time alignment and structured processing, it generates a main event sequence containing event identifiers and decision trigger identifiers. The state storage module stores historical policy branch states and policy branch states to be evaluated. The replay execution module, connected to the state storage module, inputs the same external event read from the main event sequence into two policy branch states. When the decision trigger identifier is read, it pauses event progression and determines the environment state based on the processed events. When the decision trigger identifier is not read, it continues to advance the two policy branch states. The policy adaptation interface is used to input the resource request event and the environment state as fixed parameters for the reinforcement learning scheduling policy to be evaluated, and receives the scheduling actions to be evaluated. The evaluation module updates the two policy branch states based on the historical scheduling actions in the historical scheduling logs and the scheduling actions to be evaluated. It associates the execution results and evaluation metrics with the event identifiers and performs pairing comparisons to generate policy evaluation results.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, multi-source operational metrics, container platform events, and historical scheduling logs are time-aligned and structured to form a master event sequence with event identifiers and decision trigger identifiers, ensuring that resource requests, environment states, and historical scheduling actions maintain event-level correlation. Historical policy branch states and the branch states of policies to be evaluated receive the same external events and are updated according to their corresponding scheduling actions, avoiding cross-transmission of state changes between different policies. Reinforcement learning scheduling policies with fixed parameter versions receive the same resource requests and environment states at the decision trigger position. Combined with event identifiers, the action execution results and evaluation metrics are paired and compared, ensuring that policy evaluation is based on consistent input conditions and independent state evolution, and supporting the traceability of evaluation results to specific resource request events. Attached Figure Description

[0016] Figure 1 This is a flowchart of the intelligent scheduling strategy simulation and evaluation method based on multi-dimensional time series data playback according to the present invention. Figure 2 This is a flowchart of the multi-source record time alignment and main event sequence generation process of the present invention; Figure 3 This is a data flow diagram illustrating the dual-branch state establishment and event progression of the present invention. Figure 4 This invention generates a logic diagram for decision-making pause boundaries and environmental states. Figure 5 This is a data flow diagram for the action update and event pairing evaluation of the present invention. Detailed Implementation

[0017] The following implementation methods, based on the scope of protection text and the foregoing invention content, describe the feasible process of the present invention. To maintain terminology consistency, heterogeneous computing device operation indicators refer to records reflecting the online status, total resources, allocated resources, available resources, device connection status, and container carrying status of heterogeneous computing devices; container platform events refer to resource requests, container lifecycle changes, and events related to changes in resource usage; historical scheduling logs refer to logs recording resource request identifiers, action recording times, target device identifiers, resource allocation amounts, scheduling execution types, scheduler confirmation status, and cancellation status. The main event sequence is a set of events formed after unified time base conversion, stable sorting, and unified field conversion. Event identifiers are used to associate events, action execution results, and evaluation indicators formed at different processing stages of the same resource request, while decision trigger identifiers indicate whether the current resource request event triggers a scheduling decision. The historical policy branch state and the policy branch state to be evaluated respectively carry the device resources, container lifecycle, and request queue status after the historical scheduling actions and the scheduling actions to be evaluated. Both start from the same baseline state snapshot and receive the same external events.

[0018] Example 1 Please see Figures 1 to 5 This embodiment provides a simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback. This embodiment uses heterogeneous computing device operating indicators, container platform events, and historical scheduling logs as inputs. It uses the action execution results and evaluation indicators of the same resource request event in the historical strategy branch state and the strategy branch state to be evaluated as pairing objects. Through unified timing, unified initial state, unified external events, and decision-time boundary control, the two strategy branches are allowed to advance separately under corresponding event conditions. The method includes five main steps, S1 to S5. The output of each main step is received by subsequent main steps, forming a closed processing chain from the original record input to the strategy evaluation result output.

[0019] S1 collects operating indicators of heterogeneous computing devices, container platform events, and historical scheduling logs, and generates a main event sequence containing event identifiers and decision trigger identifiers after time alignment and structured processing.

[0020] S111 retrieves raw records from the heterogeneous computing device monitoring interface, container platform interface, and scheduler log interface, respectively, and extracts the collection time, occurrence time, data source identifier, and raw record sequence number from each raw record. The collection time indicates the time the corresponding interface receives the raw record; the occurrence time indicates the actual time when the running state, platform event, or scheduling action described in the raw record actually occurs; the data source identifier distinguishes heterogeneous computing device operating indicators, container platform events, and historical scheduling logs; and the raw record sequence number maintains the order within the same data source. All time representations are converted to a unified time base, and the converted occurrence time, collection time, data source identifier, and raw record sequence number together constitute a unified timestamp record. Only records with a definite occurrence time, data source identifier, and raw record sequence number are included in the subsequent stable sorting; when the same raw record is received repeatedly, the data source identifier and raw record sequence number identify duplicates to prevent the same record from being repeatedly written into the main event sequence.

[0021] S112 performs a stable sorting of the unified timestamp records according to their occurrence time, data source priority, and original record sequence number. Records with earlier occurrence times are placed first; when occurrence times are within the same alignment window, the order of records from different data sources is determined according to a pre-fixed data source priority; when both occurrence time and data source priority are the same, the internal order of the data source is maintained according to the original record sequence number. The data source priority is fixed before the start of a playback and remains unchanged during that playback, ensuring that the same group of original records forms the same alignment event record when executed repeatedly. The input to the stable sorting is the unified timestamp records, and the output is the alignment event records with a defined order, which is received by S113.

[0022] S113, convert the alignment event records into unified fields. A unified field must at least contain the occurrence time, collection time, data source identifier, original record sequence number, event type, and content related to resources, containers, or scheduling actions in the original record. For alignment event records of resource request types, an event identifier is written based on the resource request identifier and its occurrence time. For historical scheduling actions associated with this resource request, an association is established based on the resource request identifier and the action record time, and an event identifier consistent with the corresponding resource request event is written. When a resource request event satisfies the type relationship that triggers a scheduling decision, a valid decision trigger identifier is written; external events that do not trigger scheduling decisions remain in a non-triggered state. After the above field conversion, a main event sequence is formed. The main event sequence serves as the common event input for both the historical strategy branch state and the strategy branch state to be evaluated. By checking whether the same input record forms the same sorting position, event identifier, and decision trigger identifier, the deterministic nature of the output of this step can be verified.

[0023] S2, establish historical strategy branch states and strategy branch states to be evaluated based on the main event sequence, and input the same external events into the historical strategy branch states and strategy branch states to be evaluated.

[0024] S211: Read the occurrence time of the first external event in the main event sequence, and obtain the device resource topology, total node resources, allocated node resources, container lifecycle status, and pending scheduling request queue of the last recorded event before that occurrence time. Associate status records from different data sources according to resource object identifiers, delete status records whose occurrence time is later than the occurrence time of the first external event, and form candidate status records. Device resource topology is used to determine the resource ownership relationship between heterogeneous computing devices and nodes; total node resources and allocated node resources are used to determine the available resources of nodes; container lifecycle status is used to determine whether containers have already occupied resources; and the pending scheduling request queue is used to identify resource requests that have not yet been scheduled before the start of playback.

[0025] S212, perform consistency verification on candidate state records. For each node, compare the node's allocated resources with the corresponding total node resources; if the node's allocated resources do not exceed the total node resources, and the container lifecycle state is consistent with the resource occupancy relationship, retain the candidate state record. If the node's allocated resources exceed the total node resources, or if the resource occupancy reflected by the container lifecycle state is inconsistent with the node's allocated resources, select the record from the prior state records of the same resource object whose occurrence time is closest and earlier than the first external event, and replace it. After replacement, check again according to the same relationship to form a valid state record. Subsequently, solidify the valid state record according to the fixed field order of device resource topology, node available resources, container lifecycle state, and pending scheduling request queue, write the state version number and snapshot generation time, and generate a baseline state snapshot. The baseline state snapshot does not include state changes caused by the first external event and subsequent events, and it can be verified through the consistency relationship between the total node resources, the node's allocated resources, and the container resource occupancy.

[0026] S213, the baseline state snapshot is copied to two isolated state namespaces: a first state namespace and a second state namespace. The first state namespace is written with the historical policy branch identifier, and the second state namespace is written with the policy branch identifier to be evaluated. The same baseline state snapshot version number is recorded in both state namespaces. Mutable states are not shared between the two state namespaces; resource deductions, container state transitions, or request queue removals in either branch are not directly written to the other branch. The event cursors for both the historical policy branch state and the policy branch state to be evaluated are positioned before the first external event. During replay execution, an external event is read from the main event sequence, written to both policy branches, and the corresponding event cursor is moved. When the external event does not have a valid decision trigger identifier, both policy branches continue to advance according to the same external event. Thus, the initial state version, event progression starting point, and external event input of the two policy branches remain consistent, while the state changes caused by policy actions are isolated from each other.

[0027] S3: When a resource request event with a decision trigger identifier is read, the event progression is paused, the environment state is determined based on the processed events, and the resource request event and the environment state input parameters are fixed to the reinforcement learning scheduling policy to be evaluated, and the scheduling action to be evaluated is obtained; when no such event is read, the historical policy branch state and the policy branch state to be evaluated are continued to be advanced.

[0028] S311: Read the decision trigger flag value, request lifecycle state, and event flag of the current resource request event. When the decision trigger flag value is valid and the request lifecycle state is in the pending scheduling state, record the current event cursors of the historical policy branch state and the policy branch state to be evaluated, respectively, and prohibit reading external events that occurred later than the current resource request event, generating a pause boundary record. The pause boundary record corresponds at least to the current event flag, the occurrence time of the current resource request event, and the current event cursors of the two policy branches. During the pause, device state changes, container platform events, and historical scheduling log records located after the pause boundary in the main event sequence do not enter the environment state used for the current decision. When the decision trigger flag value is invalid, or the request lifecycle state is not in the pending scheduling state, no scheduling decision is formed for the current resource request, and the two policy branches continue to advance according to the main event sequence.

[0029] S312: Obtain the available node resources, container resource usage, pending scheduling request queue, and device connection status from the branch state of the policy to be evaluated before the pause boundary record, excluding state changes caused by the current resource request event itself and events that occurred later than the pause boundary record. Simultaneously, obtain the computing resource requirements, storage resource requirements, device type constraints, and container deployment constraints from the current resource request event to form request constraint data; and obtain the online status, available resources, and container carrying status of all heterogeneous computing devices from the branch state of the policy to be evaluated to form device status data.

[0030] S313, filter device status data based on request constraint data. First, retain heterogeneous computing devices with valid online status; then, retain heterogeneous computing devices whose device type meets the device type constraints; next, compare the available resources of the devices with the computing resource requirements and storage resource requirements respectively, and only retain heterogeneous computing devices whose available resources are not lower than the corresponding requirements; finally, exclude heterogeneous computing devices with deployment conflicts based on container deployment constraints, forming a candidate device record. Each filtering condition applies to the previous filtering result in the above order, and devices that do not meet any condition are not included in the candidate device record. Arrange the candidate device records in a fixed order according to the device identifier, and determine the remaining resource amount, number of running containers, number of request queues, and device connection status for each candidate device, and then associate them with the request constraint data to generate the environment status. The device order and field order of the environment status are fixed in one evaluation, so that the same branch status and the same resource request event form the same structured input.

[0031] S314. The current resource request event and environment state are structured and encoded according to a fixed input field order. The resource request field in the encoded input comes from the current resource request event, and the device field comes from candidate device records arranged by device identifier. Neither includes state content formed after the pause boundary. The encoded result is input into the reinforcement learning scheduling policy to be evaluated, whose parameter version is fixed. Fixed parameter version means that the parameter version used by the policy remains unchanged during a main event sequence replay and is not updated due to the execution result of the action in the current replay. After parsing, the policy output forms a resource allocation object and a scheduling execution type, and generates the scheduling action to be evaluated accordingly. The resource allocation object should correspond to a candidate device record in the environment state; when the policy output cannot correspond to a candidate device record, the output is not written to the executable state change record, and the current resource request's state in the queue of scheduled requests remains unchanged. After the scheduling action to be evaluated is formed, the pause state corresponding to the current event is released, allowing subsequent state updates and event progression to continue.

[0032] S4, update the historical policy branch status based on the historical scheduling actions in the historical scheduling log, and update the policy branch status to be evaluated based on the scheduling actions to be evaluated.

[0033] S411, based on the event identifier, resource request identifier, and occurrence time of the current resource request event, candidate historical actions with the same resource request identifier and an action record time no earlier than the request occurrence time are retrieved from the historical scheduling log, forming candidate historical action records. When multiple candidate historical action records exist, the scheduler confirmation status is checked first, prioritizing the retention of records that the scheduler has confirmed execution, and selecting the record with the earliest action record time from among them; if no candidate record has been confirmed execution by the scheduler, records that have been revoked are excluded, and the record with the earliest action record time is selected from the records that have not been revoked. The selected records form a matching historical action record, in which the target device identifier, resource allocation amount, and scheduling execution type are determined as historical scheduling actions. This matching process uses the resource request identifier, request occurrence time, scheduler confirmation status, revocation status, and action record time to form a definite filtering order, avoiding uncertain selection when the same resource request corresponds to multiple historical actions.

[0034] S412 parses the resource request identifier, target device identifier, resource allocation amount, and scheduling execution type from historical scheduling actions and scheduling actions to be evaluated, and associates the current resource request event with the resource request identifier to form historical action records and action records to be evaluated. Historical action records only enter the historical policy branch state, and action records to be evaluated only enter the action record to be evaluated policy branch state. For each action record, the target device identifier is associated with the device online status in the corresponding policy branch state, the resource allocation amount is associated with the node's available resources, and the resource request identifier is associated with the request lifecycle state, thereby determining whether the action has the corresponding object and state basis to cause a state change.

[0035] S413, in the historical policy branch state, resource deduction, container state migration, and request queue removal are determined based on historical action records, forming a first state change record; in the policy branch state to be evaluated, state change items of the same category are determined based on the action records to be evaluated, forming a second state change record. Resource deduction targets the target device identifier and resource allocation amount in the action record; container state migration targets the container lifecycle state formed by the corresponding resource request; and request queue removal targets the pending scheduling request corresponding to the resource request identifier. Only when the target device in the action record can obtain online status and available node resources in the corresponding policy branch state, and the resource request is in a lifecycle state that can accept the scheduling action, will the corresponding item be written into the state change record; items that do not meet the matching relationship do not change the available node resources, container lifecycle state, and pending scheduling request queue in the original branch.

[0036] S414: Write the first state change record to the historical policy branch state and the second state change record to the policy branch state to be evaluated, forming the updated historical policy branch state and the updated policy branch state to be evaluated, respectively. After writing, the action execution status is recorded with an event identifier, and the resource deduction result, container lifecycle state, and request queue state are kept consistent. The write operations of the two branches act on the first state namespace and the second state namespace, respectively, so different node available resources and container carrying states can be formed after the same external event. After the update is completed, the event cursors of the two branches continue to read subsequent external events in the main event sequence from the pause position. The subsequent environment state is generated only from the processed content of the policy branch state to be evaluated before the corresponding pause boundary.

[0037] S5 uses event identifiers to associate the action execution results and evaluation metrics of historical strategy branch states and strategy branch states to be evaluated, respectively. It then pairs and compares the action execution results and evaluation metrics corresponding to the same event identifier to generate strategy evaluation results.

[0038] S511: For each resource request event, based on the event identifier, obtain the action execution status, resource request completion status, and evaluation index value from the historical policy branch status and the policy branch status to be evaluated, respectively. The action execution status reflects whether the corresponding scheduling action has formed a status change record and been written to the corresponding policy branch. The resource request completion status reflects the lifecycle result of the resource request in the corresponding policy branch. The evaluation index value is the index data used to evaluate the scheduling result, calculated based on the action execution result, resource request completion status, and system operation status of the corresponding policy branch during the event replay process. A one-to-one correspondence is established between the results obtained from two policy branches that share the same event identifier, forming an event pairing record. If either branch has not formed a corresponding action execution result, its action execution status and resource request completion status are still retained using the same event identifier, ensuring that the pairing relationship is not lost due to the action not being executed.

[0039] S512, within each event pairing record, compare the historical metric values ​​and the metric values ​​to be evaluated for the same evaluation indicator. Simultaneously, consider the action execution status and resource request completion status of the historical strategy branch states, as well as the action execution status and resource request completion status of the strategy to be evaluated branch state, to determine the single-event comparison result. Comparisons are only performed between the same event identifier and the same evaluation indicator category; values ​​from different resource request events or different indicator categories are not substituted for each other. The single-event comparison result indicates the comparison relationship between the strategy to be evaluated and historical strategies under the current resource request event. Its input is the event pairing record, and its output is summarized by S513.

[0040] S513: Summarize multiple single-event comparison results according to evaluation indicator categories, and count the number of events in which the evaluated strategy is superior to, equal to, and inferior to historical strategies, respectively, generating strategy evaluation results. The summarization process retains the correspondence between event identifiers and single-event comparison results, ensuring that any summarized quantity can be traced back to the corresponding resource request event, the action execution status in the two strategy branches, the resource request completion status, and the evaluation indicator value. By checking the correspondence between the number of event pairing records and the number of resource request events entering the evaluation, and by checking whether the three types of event quantities all come from the formed single-event comparison results, the data source and summarization path of the strategy evaluation results can be verified.

[0041] During the replay process in this embodiment, the main event sequence only provides common external events and matching historical actions from the historical scheduling log, without directly writing changes in the branch state of the policy to be evaluated into the historical policy branch state. Each time a valid decision trigger flag is encountered, the range of data that can enter the environment state is limited by pausing boundary records, preventing node states, container states, or device connection states formed by subsequent events from prematurely participating in the current scheduling decision. The baseline state snapshot version, main event sequence sorting rules, candidate device arrangement order, policy input field order, and parameter version of the reinforcement learning scheduling policy to be evaluated remain fixed throughout a single evaluation, thereby enabling the same input conditions to form a reproducible event progression, action generation, and pairing comparison process.

[0042] Example 2 Please see Figure 1 , Figures 3 to 5 This embodiment provides an intelligent scheduling strategy simulation and evaluation system based on multi-dimensional time-series data playback. The difference between this embodiment and Embodiment 1 is that Embodiment 1 focuses on method steps and data state changes, while this embodiment implements the aforementioned method through the connections, calls, and data reception relationships between the data processing module, state storage module, playback execution module, strategy adaptation interface, and evaluation module. The data names, event boundaries, and state update rules received and output by each unit remain consistent with those in Embodiment 1.

[0043] The data processing module establishes data receiving relationships with the heterogeneous computing device monitoring interface, container platform interface, and scheduler log interface, respectively. The heterogeneous computing device monitoring interface sends heterogeneous computing device operation indicators to the data processing module, the container platform interface sends container platform events to the data processing module, and the scheduler log interface sends historical scheduling logs to the data processing module. The data processing module extracts the collection time, occurrence time, data source identifier, and original record sequence number from the received records, converts the time expression to a unified time base, and performs stable sorting according to occurrence time, data source priority, and original record sequence number. The sorted aligned event records are then converted using unified fields and written with event identifiers and decision trigger identifiers to form the main event sequence. The data processing module outputs the main event sequence to the playback execution module, enabling the action records in the historical scheduling logs to be invoked by the evaluation module according to resource request identifiers, event identifiers, and time relationships.

[0044] The state storage module comprises a first state namespace and a second state namespace, which are isolated from each other. The first state namespace stores historical policy branch states, while the second state namespace stores the policy branch states to be evaluated. The state storage module receives a baseline state snapshot and writes snapshots of the same version of the baseline state to both state namespaces. Each state namespace stores at least the device resource topology, available node resources, container lifecycle states, queue of scheduled requests, branch identifiers, state version numbers, and event cursors. The replay execution module and the evaluation module both carry the policy branch identifiers when writing to the state storage module, ensuring that first state change records are only written to the first state namespace, and second state change records are only written to the second state namespace.

[0045] The replay execution module is connected to the data processing module and the state storage module. The replay execution module reads the current external event from the main event sequence, inputs the same current external event into the historical policy branch state and the policy branch state to be evaluated, and obtains the event cursors for both policy branches. When the current external event does not have a valid decision trigger identifier, the replay execution module continues to advance the two event cursors; when the current external event has a valid decision trigger identifier and the request lifecycle state is in the pending scheduling state, the replay execution module records the occurrence time of the current event cursor and the current resource request event, forming a pause boundary record, and stops reading external events whose occurrence time is later than the current resource request event. The replay execution module reads the available resources of the node, container resource usage, pending scheduling request queue, and device connection status before the pause boundary from the second state namespace, and combines this with the computing resource requirements, storage resource requirements, device type constraints, and container deployment constraints in the current resource request event to form the environment state.

[0046] The policy adaptation interface receives the current resource request event and environment state as input. It forms a structured code according to a fixed input field order, sends this structured code to the reinforcement learning scheduling policy to be evaluated (with fixed parameter versions), and receives the resource allocation object and scheduling execution type returned by the policy. The policy adaptation interface parses the returned content that corresponds to a candidate device record into a scheduling action to be evaluated and outputs this action to the evaluation module. If the policy's returned content does not correspond to a candidate device record, the policy adaptation interface does not output a scheduling action to be evaluated and returns or retains the pending scheduling status of the resource request to the replay execution module. This returned status allows the replay execution module to continue processing subsequent events without writing an action indicating no corresponding device.

[0047] The evaluation module receives historical scheduling actions and scheduling actions to be evaluated. For historical scheduling actions, the evaluation module determines matching historical action records from the historical scheduling log based on the resource request identifier, request occurrence time, scheduler confirmation status, cancellation status, and action recording time. For scheduling actions to be evaluated, the evaluation module receives the resource request identifier, target device identifier, resource allocation amount, and scheduling execution type generated by the policy adaptation interface. The evaluation module matches the two types of actions with the device online status, node available resources, and request lifecycle status in the corresponding status namespace, forming a first status change record and a second status change record, and writes them to the status storage module. After writing, the evaluation module obtains the action execution status, resource request completion status, and evaluation index value in the two policy branches based on the event identifier, forming an event pairing record, single event comparison results, and policy evaluation results.

[0048] The data processing module transmits the main event sequence to the playback execution module; the playback execution module transmits external events, event cursors, and pause boundary records to the state storage module; the state storage module returns basic environmental state data from the branch state of the policy to be evaluated to the playback execution module; the playback execution module transmits the current resource request event and environmental state to the policy adaptation interface; the policy adaptation interface transmits the scheduled action to be evaluated to the evaluation module; and the evaluation module and the state storage module transmit state change records and the updated policy branch state. The policy evaluation result generated by the evaluation module is the system output. The above data flow allows us to examine the complete transmission path of any event identifier from the generation of the main event sequence, the association of policy actions, the update of branch state, to the evaluation pairing.

[0049] The specific processes, field states, processing order, judgment conditions, state namespaces, data transmission relationships, and unit collaboration relationships described in the above embodiments are used to illustrate the possible implementations of the present invention and should not be construed as limiting the present invention to the specific embodiments listed. Without departing from the scope of the technical solutions described in the present invention, any equivalent substitutions, modifications, combinations, order adjustments, unit correspondence replacements, equivalent transformations of field names, equivalent acceptance of the executing entity, or equivalent changes in the carrier form that can be made by those skilled in the art should fall within the scope of protection of this patent; however, they should not be extended to protected subjects not recorded in the application documents, nor should the substantive correspondence between technical objects be changed through changes in name or identifier.

Claims

1. A simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback, characterized in that, include: Collect heterogeneous computing device operation indicators, container platform events, and historical scheduling logs, and generate a main event sequence containing event identifiers and decision trigger identifiers after time alignment and structured processing; Based on the main event sequence, establish historical strategy branch states and strategy branch states to be evaluated, and input the same external events into the historical strategy branch states and strategy branch states to be evaluated; When a resource request event with the decision trigger identifier is read, the event progression is paused, the environment state is determined based on the processed events, and the resource request event and the environment state are used to input parameters of the fixed reinforcement learning scheduling policy to be evaluated to obtain the scheduling action to be evaluated; when no such event is read, the historical policy branch state and the policy branch state to be evaluated are continued to be advanced. Update the historical policy branch status based on the historical scheduling actions in the historical scheduling log, and update the policy branch status to be evaluated based on the scheduling actions to be evaluated; The event identifier is used to associate the action execution results and evaluation indicators of the historical strategy branch state and the strategy branch state to be evaluated, respectively. The action execution results and evaluation indicators corresponding to the same event identifier are paired and compared to generate strategy evaluation results.

2. The simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback according to claim 1, characterized in that, The process of generating the main event sequence includes: Obtain the original records of the heterogeneous computing device operation indicators, the container platform events, and the historical scheduling logs; extract the collection time, occurrence time, data source identifier, and original record sequence number; and convert each time expression to a unified time base to generate a unified timestamp record. The unified timestamp records are stably sorted according to the occurrence time, data source priority, and original record sequence number. When multiple occurrence times are in the same alignment window, a deterministic order is maintained according to the data source priority to generate aligned event records. The alignment event record is converted into a unified field, and the event identifier and the decision trigger identifier are written according to the resource request type and the historical scheduling action association, respectively, to generate the main event sequence.

3. The simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback according to claim 1, characterized in that, The process of establishing the historical strategy branch state and the strategy branch state to be evaluated includes: Obtain the device resource topology, node available resources, container lifecycle status, and pending scheduling request queue before the occurrence of the first external event in the main event sequence, and remove status records with timestamps later than the occurrence time of the first external event to generate a baseline status snapshot; Copy the baseline state snapshot to a first state namespace and a second state namespace that are isolated from each other, write the historical strategy branch identifier and the strategy branch identifier to be evaluated into them respectively, freeze the version number of the baseline state snapshot, and generate the historical strategy branch state and the strategy branch state to be evaluated. Position the event cursors of both the historical strategy branch state and the strategy branch state to be evaluated before the first external event, so that the historical strategy branch state and the strategy branch state to be evaluated have the same starting point for event progression.

4. The simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback according to claim 3, characterized in that, The process of generating the baseline state snapshot specifically includes: Obtain the device resource topology, total node resources, allocated node resources, container lifecycle status, and pending scheduling request queue recorded last before the occurrence of the first external event. Associate the status records in different data sources according to the resource object identifier, delete the status records whose occurrence time is later than the occurrence time of the first external event, and generate candidate status records. Compare the total node resources in the candidate state records with the node's allocated resources. If the node's allocated resources do not exceed the corresponding node's total resources and the container's lifecycle state is consistent with the resource occupancy relationship, retain the corresponding candidate state record. If they are inconsistent, replace the previous state record that occurred closest to and earlier than the first external event to generate a valid state record. The valid status records are solidified according to the fixed field order of device resource topology, node available resources, container lifecycle status, and pending scheduling request queue, and the status version number and snapshot generation time are written to generate the baseline status snapshot.

5. The simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback according to claim 1, characterized in that, The process of reading the resource request event with the decision trigger identifier and obtaining the scheduling action to be evaluated includes: Read the decision trigger identifier, request lifecycle state, and event identifier of the current resource request event. When the decision trigger identifier value is valid and the request lifecycle state is in the pending scheduling state, record the current event cursors of the historical strategy branch state and the strategy branch state to be evaluated, and prohibit reading external events that occurred later than the current resource request event, and generate a pause boundary record. The available resources of the node, container resource usage, pending scheduling request queue and device connection status of the branch state of the strategy to be evaluated before the pause boundary record are obtained, and the state changes caused by the current resource request event and events that occurred later than the pause boundary record are excluded to generate the environment state; The current resource request event and the environment state are structured and encoded according to a fixed input field order. The encoded result is input into the reinforcement learning scheduling policy to be evaluated with a fixed parameter version. The resource allocation object and scheduling execution type output by the policy are received and parsed to generate the scheduling action to be evaluated.

6. The simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback according to claim 5, characterized in that, The process of generating the environmental state specifically includes: The system obtains the computing resource requirements, storage resource requirements, device type constraints, and container deployment constraints from the current resource request event, and obtains the online status, available resources, and container carrying status of all heterogeneous computing devices from the branch status of the strategy to be evaluated, and generates request constraint data and device status data. From the device status data, select heterogeneous computing devices that are online and valid, whose device type meets the device type constraints, and whose available resources are not less than the computing resource requirements and the storage resource requirements. Based on the container deployment constraints, exclude heterogeneous computing devices with deployment conflicts and generate candidate device records. The candidate device records are arranged in a fixed order according to the device identifier. The available node resources, container resource usage, scheduling request queue and device connection status of each candidate device are determined. The determination results are then associated with the request constraint data to generate the environment state.

7. The simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback according to claim 1, characterized in that, The process of updating the historical strategy branch state and the strategy branch state to be evaluated includes: The resource request identifier, target device identifier, resource allocation amount, and scheduling execution type in the historical scheduling actions and the scheduling actions to be evaluated are parsed respectively, and the corresponding resource request events are associated with the resource request identifier to generate historical action records and action records to be evaluated. The historical action records and the action records to be evaluated are matched with the device online status, node available resources and request lifecycle status in the corresponding strategy branch status to determine resource deduction, container state migration and request queue removal, and generate the first state change record and the second state change record. Write the first state change record into the historical strategy branch state, and write the second state change record into the strategy branch state to be evaluated, thereby generating the updated historical strategy branch state and the updated strategy branch state to be evaluated, respectively.

8. The simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback according to claim 1, characterized in that, The process of determining the historical scheduling actions includes: Based on the event identifier, resource request identifier, and occurrence time of the current resource request event, candidate historical actions with the same resource request identifier and an action recording time no earlier than the request occurrence time are selected from the historical scheduling log, and candidate historical action records are generated. When multiple candidate historical action records exist, the candidate historical action record with the earliest confirmed execution time in terms of available node resources, container resource usage, and pending scheduling request queue is selected first. When no candidate historical action record with confirmed execution in terms of available node resources, container resource usage, and pending scheduling request queue is available, the candidate historical action record with the earliest execution time and not revoked is selected to generate a matching historical action record. The target device identifier, resource allocation amount, and scheduling execution type in the matching historical action record are determined as the historical scheduling action.

9. The simulation and evaluation method for intelligent scheduling strategies based on multi-dimensional time-series data playback according to claim 1, characterized in that, The process of generating the policy evaluation results includes: Based on the event identifier of each resource request event, the action execution status, resource request completion status and historical evaluation index value are obtained from the historical strategy branch status, and the action execution status, resource request completion status and evaluation index value to be evaluated are obtained from the strategy branch status to be evaluated. The two sets of obtained results belonging to the same event identifier are established to establish a corresponding relationship and generate an event pairing record. The historical index values ​​and the index values ​​to be evaluated for the same evaluation index in the event pairing records are compared respectively. The single event comparison result for each resource request event is determined by combining the action execution status and resource request completion status of the historical strategy branch state and the action execution status and resource request completion status of the strategy branch state to be evaluated. The evaluation results are generated by summarizing the comparison results of multiple single events according to the evaluation index categories, and counting the number of events in which the strategy under evaluation is better than, equal to, or worse than the historical strategy.

10. A simulation and evaluation system for intelligent scheduling strategies based on multi-dimensional time-series data playback, used to support the method described in any one of claims 1 to 9, characterized in that, include: The data processing module receives heterogeneous computing device operation indicators, container platform events, and historical scheduling logs through the heterogeneous computing device monitoring interface, container platform interface, and scheduler log interface. After time alignment and structured processing, it generates a main event sequence containing event identifiers and decision trigger identifiers. The state storage module saves the historical strategy branch states and the strategy branch states to be evaluated. The replay execution module, connected to the state storage module, inputs the same external event read from the main event sequence into the historical strategy branch state and the strategy branch state to be evaluated, respectively. When a resource request event with the decision trigger identifier is read, the event progression is paused and the environment state is determined based on the processed events. When the decision trigger identifier is not read, the historical strategy branch state and the strategy branch state to be evaluated continue to be progressed. The strategy adaptation interface receives the resource request event with the decision trigger identifier and the reinforcement learning scheduling strategy to be evaluated with the fixed environmental state input parameters, and receives the scheduling action to be evaluated. The evaluation module updates the historical policy branch status and the policy branch status to be evaluated based on the historical scheduling actions in the historical scheduling log and the scheduling actions to be evaluated, respectively. It associates the action execution results and evaluation indicators with the event identifiers and compares them to generate policy evaluation results.

Images