A heterogeneous distributed node pre-scheduling and hardware snapshot migration system based on GB / T 47518 metadata

Abstract

The application discloses a heterogeneous distributed node pre-scheduling and hardware snapshot migration system based on GB / T 47518 metadata, belongs to the technical field of distributed computing power scheduling, and is based on a pre-scheduling system of GB / T 47518 full fields. A field-level line-by-line hardware comparison admission mechanism is created, which is different from traditional fuzzy matching, national standard level accurate hardware adaptation is realized, hardware snapshot 1:1 environment replication migration solves the core pain point of'migration degradation' which has not been solved in the industry for ten years, and a large-scale cluster national standard metadata lightweight incremental governance architecture is created to solve the problems of bloated national standard data, high delay and update distortion of ten thousand nodes.

Description

Technical Field

[0001] This invention relates to the fields of distributed heterogeneous computing power scheduling, standardized governance of computing resources, fault-tolerant migration of edge cloud cluster tasks, and standardized metadata parsing and incremental update technology. Specifically, it relates to a heterogeneous distributed node pre-scheduling and hardware snapshot migration system based on GB / T 47518 metadata. Background Technology

[0002] With the full implementation of the national integrated computing power network and the East-West computing project, the number of heterogeneous computing power nodes across the country has exploded. The state has issued the mandatory standard GB / T 47518-2026 to unify the structured XML output format of hardware descriptions, computing power units, storage, network, and reliability parameters of all computing power nodes.

[0003] There are four major unsolvable loopholes in the implementation of national standards in the current industry technology: Existing computing power scheduling methods are all coarse-grained scheduling methods. There is no national standard-level field-by-field automatic hardware compatibility verification mechanism. Node failure migration commonly results in performance degradation. Large-scale cluster national standard metadata is bloated and updates are lagging. Loading tens of thousands of nodes with full XML results in high latency and overhead, and hardware changes cannot be incrementally updated, leading to distorted scheduling decision data. Attached Figure Description

[0004] Figure 1 This is a schematic diagram illustrating the overall architecture of a heterogeneous distributed node pre-scheduling and hardware snapshot migration system based on GB / T 47518 metadata, according to the present invention.

[0005] Figure 2 This is a schematic diagram of the workflow for the national standard metadata parsing and lightweight governance module.

[0006] Figure 3 A schematic diagram of the workflow of the national standard full-dimensional hardware profiling and dynamic weight matching scheduling module.

[0007] Figure 4 A schematic diagram of the workflow for the pre-deployment verification module for national standard field-level hardware compatibility access.

[0008] Figure 5 This is a schematic diagram of the workflow of a national standard hardware snapshot-type fault-tolerant module with no degradation migration. Summary of the Invention

[0009] Core technologies: A dynamic task-adaptive computing power profiling mechanism based on national standard full-dimensional hardware fields.

[0010] A cross-node hardware compatibility access verification engine that compares each field of the national standard XML message.

[0011] A non-degradable task fault migration architecture based on a complete snapshot of national standard hardware.

[0012] A lightweight incremental governance architecture for national standard computing power metadata for large-scale clusters.

[0013] This invention enables precise screening before task assignment, fully automatic compliance verification upon network access, lossless migration of faults during operation, and efficient governance of massive node metadata, filling the gap in national standards that only define resource description formats.

[0014] Overall process Each distributed node automatically generates compliant GB / T 47518 XML metadata files. The system is lightweight, performs quantitative parsing, incremental updates, and builds a global hardware index.

[0015] 3. When a new task enters the cluster, the system identifies the task type.

[0016] 4. Dynamically load the national standard five-dimensional hardware weight model.

[0017] 5. Match the optimal computing power node based on the feature vector of the national standard field.

[0018] 6. Perform a field-by-field hardware compatibility verification on the selected node.

[0019] 7. Automatically generate container configurations and complete pre-deployment.

[0020] 8. Continuously bind to national standard hardware snapshots during task execution.

[0021] 9. In the event of a node failure, a backup node of the same specification is matched based on the snapshot.

[0022] 10. Replicate the hardware environment to complete a migration without downgrades.

[0023] 11. Record all changes to national standard metadata throughout the process for platform compliance traceability.

Claims

1. A heterogeneous distributed node pre-scheduling and hardware snapshot migration system based on GB / T 47518 metadata, characterized in that, include: The system comprises four modules: a lightweight incremental governance module for GB / T 47518-2026 computing power metadata, a dynamic matching and scheduling module for GB / T 47518-2026 full-dimensional hardware profiles, a pre-deployment module for GB / T 47518-2026 field-level hardware compatibility access, and a fault migration module for GB / T 47518 hardware snapshots. The lightweight incremental governance module parses, compresses, and incrementally updates cpdrd namespace XML computing power metadata conforming to GB / T 47518-2026, distinguishes between core and extended scheduling fields, and constructs a distributed sharding index. The dynamic matching and scheduling module generates adaptive weights for different task types based on the five dimensions of processor, memory, storage, network, and hardware reliability defined in GB / T 47518, constructs hardware feature vectors, and completes pre-matching and screening of nodes. The pre-deployment module automatically compares and verifies task hardware dependency parameters against node GB / T 47518 metadata, automatically generates container resource configurations, completes task pre-deployment, and outputs a GB / T compliant network access report. The fault migration module continuously binds the complete GB / T 47518 data to the node during task execution. 47518 Hardware Snapshot: When a node fails, the system accurately matches a backup node of the same specification based on the hardware snapshot and completely replicates the hardware operating environment to achieve task migration without performance degradation. The system only performs task pre-scheduling, network access verification, operation fault tolerance, and metadata governance, and does not perform computing power metering, computing power conversion, SCU calculation, AHP weight calculation, operator statistics, and computing power settlement services.

2. The system according to claim 1, characterized in that: The lightweight incremental governance module for national standard computing power metadata includes an XML hierarchical parsing subunit, a core field caching subunit, a hardware change incremental update subunit, and a distributed sharding index subunit. It only performs resident memory processing on fields required for computing power scheduling as specified by the national standard, and loads extended remarks fields as needed.

3. The system according to claim 1, characterized in that: The national standard full-dimensional hardware profile dynamic matching and scheduling module has built-in independent weight models for four types of tasks: large model training, AI inference, simulation calculation, and general compilation. It dynamically adjusts the weight coefficients of each dimension according to the national standard hardware parameters to generate a unique hardware adaptation feature vector.

4. The system according to claim 1, characterized in that: The national standard field-level hardware compatibility access pre-deployment module automatically compares the national standard field parameters of CPU instruction set, GPU interface version, FPGA logic resources, storage RAID architecture, and network protocol stack to achieve fully automatic compliance verification for cross-vendor heterogeneous nodes.

5. The system according to claim 1, characterized in that: The national standard hardware snapshot non-degradation fault migration module binds a complete national standard XML hardware snapshot throughout the task execution process. During fault migration, the cache level, memory bandwidth, storage IO strategy, and network forwarding parameters are replicated 1:1 according to the snapshot parameters to ensure that the task execution accuracy and throughput are not reduced.

6. The system according to claim 1, characterized in that: The data sources and judgment criteria for all scheduling logic, matching logic, verification logic, and migration logic in the system are strictly taken from the computing power resource description fields of GB / T 47518-2026 standard.

Images