An AI computing power scheduling method and system for a private large model

Abstract

This invention relates to the field of computing power scheduling technology, specifically to an AI computing power scheduling method and system for private large-scale models. The invention performs attention encoding on multi-source heterogeneous data to obtain format recognition results and acquisition priority labels; generates a memory pressure pre-transmission signal based on the format recognition results; extracts structural features from high-quality data streams and generates computing power affinity encoding vectors through graph neural networks; constructs a state space combined with a secure computing power pre-occupancy mechanism, outputting a resource allocation action sequence; further performs memory oversubscription, core binding, NUMA node crossing, and network bandwidth conflict detection, and resolves conflicts and performs affinity rearrangement based on task priorities to generate a conflict-free instruction sequence. This invention can improve the utilization efficiency of heterogeneous computing power resources, reduce the probability of resource conflicts, and improve the scheduling stability and execution efficiency during the training and inference process of private large-scale models.

Description

Technical Field

[0001] This invention relates to the field of computing power scheduling technology, specifically to an AI computing power scheduling method and system for private large-scale models. Background Technology

[0002] With the large-scale deployment of large-scale model technology, industries with high data security requirements, such as finance, government, and healthcare, generally adopt private deployment to run large-scale model platforms. In private deployment scenarios, large-scale model platforms need to support both model training and real-time inference tasks simultaneously. The competition between these two tasks for GPU memory creates a continuous resource conflict. Moreover, unlike public cloud environments, private environments are constrained by physical hardware limits, making it impossible to alleviate resource shortages through elastic scaling. Improper resource scheduling will directly lead to interruptions inference tasks or significant regressions in training progress.

[0003] At the data acquisition level, the types of data sources within an enterprise in a private environment are diverse, including various formats such as text, images, audio, video, and structured data. The data interface standards of different business systems are not uniform, and existing acquisition solutions are difficult to achieve unified and efficient acquisition of multi-source heterogeneous data. At the same time, the acquisition nodes are deployed in different physical locations within the enterprise intranet, and the network topology between the nodes is asymmetrical and fixed. When an acquisition node fails, the existing fault tolerance mechanism cannot select the optimal replacement node based on the actual network transmission quality, and the data continuity during the failure period is also difficult to guarantee.

[0004] At the computing power scheduling level, existing scheduling systems are designed for public cloud environments. Scheduling decisions rely solely on the current utilization rate of computing resources, failing to anticipate the pressure on GPU memory from upcoming training data batches. Furthermore, they cannot include the CPU computing power consumed by encryption and desensitization operations during data security protection in the scheduling resource ledger, resulting in a systematic discrepancy between scheduling decisions and the actual available resources in the system. In addition, because the scheduler cannot predict the risk of resource exhaustion in the next scheduling cycle, it can only passively trigger emergency migration when resources are actually exhausted, leading to a high probability of forced interruption of training tasks. Existing technologies lack predictive migration mechanisms to proactively migrate low-priority tasks to backup nodes before resource criticality points, failing to effectively reduce the probability of unplanned task interruptions due to resource contention.

[0005] To address this, a method and system for scheduling AI computing power for private large-scale models is proposed. Summary of the Invention

[0006] The purpose of this invention is to provide an AI computing power scheduling method and system for private large-scale models. By using multi-source heterogeneous data attention encoding, memory pressure pre-transmission, computing power affinity modeling, and multi-layer conflict detection, it can achieve intelligent scheduling and conflict-free resource allocation of heterogeneous computing power resources, thereby improving resource utilization and scheduling stability in private large-scale model scenarios.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A method for scheduling AI computing power for private large-scale models, comprising: Attention encoding is performed on multi-source heterogeneous data to obtain format recognition results and collection priority labels; Based on the format recognition results, the video memory usage calibration table is queried to obtain the video memory increment prediction value. The video memory increment prediction value and the current video memory status are combined to form a video memory pressure pre-transmission signal. Redundant batches are selected and filtered by priority labels to obtain high-quality data streams; the type ratio, sparsity density and sequence dimension features of the high-quality data streams are extracted and mapped into computing power affinity encoding vectors via graph neural networks; Based on the format recognition result, the encrypted computing power calibration table is queried to obtain the estimated consumption of security processing. Combined with the security computing power pre-occupancy request, the scheduler is declared and the corresponding resources are deducted. The state space is constructed with the computing power affinity encoding vector, the memory pressure pre-transmission signal and the resource status, and the resource allocation action sequence is output. The system performs memory oversubscription, processor core binding conflict, NUMA node crossing, and network bandwidth conflict detection on the resource allocation action sequence. It then resolves conflicts based on task priority scores and rearranges them according to NUMA affinity to generate a conflict-free instruction sequence.

[0008] Preferably, attention encoding is performed on multi-source heterogeneous data to obtain format recognition results and acquisition priority labels, specifically including: Field distribution features, encoding format features, and dimensional organization features are extracted from batches of multi-source heterogeneous data and combined to construct a query vector. Attention weight matching calculation is performed between the query vector and the predefined data type standard template vector, and the format recognition result is determined according to the highest matching confidence. The collection priority label is generated based on the actual consumption rate of data types and the backlog of the training queue during the current training phase as dynamic states, and the training improvement ratio as a feedback signal. The collection frequency weight and collection range of data types are adjusted, and the collection priority label is output.

[0009] Preferably, the process of acquiring the pre-transmission signal of video memory pressure is as follows: for the target processor model, enumerate the combination of each data type and each batch size, actually execute the loading operation of the corresponding data batch, collect the change in video memory usage before and after loading, and establish a video memory usage calibration table with the triplet of data type label, batch sample quantity, and single sample feature dimension scale as index. After the format recognition result is output, the calibration table is queried by data type label, current batch sample quantity, and single sample feature dimension scale to obtain the predicted value of video memory increment; based on the predicted value of video memory increment and the current real-time video memory occupied by the processor, the expected remaining video memory is obtained.

[0010] Preferably, the high-quality data stream acquisition process specifically includes: determining the transmission reliability strategy for each data batch based on the collection priority label: high-priority batches are subject to end-to-end integrity verification to ensure transmission integrity; low-priority batches are subject to a lightweight sequence number mechanism for packet loss detection and selective retransmission after transmission; batches that exceed the retransmission limit are downgraded to low-confidence batches and enter the re-collection queue. Calculate the feature similarity between the current data batch and historically collected batches; batches with feature similarity exceeding a preset threshold are marked as redundant batches and skipped from subsequent transmission; the feature library uses a sliding window mechanism to retain the feature vectors of the most recent batches; after dual processing of reliability screening and redundancy filtering, a high-quality data stream is output.

[0011] Preferably, generating the computing power affinity coding vector specifically includes: sampling the batches of data to be processed from the buffer queue of high-quality data streams, extracting three structural features for each data batch: the proportion of data type composition, the ratio of non-zero feature elements in the batch to all feature elements, and the maximum sequence length of samples in the batch; and constructing feature combinations by combining historical data with similar structural features with measured throughput efficiency benchmarks on various computing power resources. A graph structure is constructed using the batches of data to be processed in the buffer queue as nodes and the structural feature similarity between batches as edge weights. If the number of batches to be processed in the current buffer queue is insufficient to form effective neighbor relationships, feature records of historically processed batches are introduced as auxiliary reference nodes. Each node in the graph structure aggregates neighbor feature information based on edge weights, and outputs a fixed-dimensional computing power affinity encoding vector after multi-layer feature transformation. Each component in the computing power affinity encoding vector explicitly quantifies the expected throughput acceleration benefits of the current data stream for parallel intensive computing power, sequence processing computing power, and vector acceleration computing power.

[0012] Preferably, constructing the state space output resource allocation action sequence specifically includes: Benchmark tests of encryption operations were performed on the target processor model for each data type. The processor usage and processing time under each data type and data volume combination, as well as the processor overhead of secure handshake under different concurrent connection numbers, were collected. An encryption computing power calibration table was established with data type and data volume as indexes and encryption processor overhead as values. After the format recognition result is output, the encryption computing power calibration table is queried to obtain the estimated processor resource consumption for the current batch of security processing, and a security computing power pre-occupancy request is sent to the scheduler; the scheduler marks the corresponding processor resource as a security pre-occupancy status in the resource ledger and does not allocate the corresponding resource to the business task within the estimated duration of the batch security processing. The actual available resource status after security pre-allocation, the computing power affinity encoding vector, and the pre-transmission signal of video memory pressure are concatenated into a state vector; the task queue is dynamically sorted in a fine-grained manner according to three dimensions: business importance, time urgency, and resource demand scale, to form a task priority score; with the state vector as input, resource allocation actions are generated for the tasks to be scheduled, including the target computing power node, the amount of video memory requested, the number of processor cores requested, the target memory node number, and the amount of network bandwidth requested. All actions constitute a resource allocation action sequence.

[0013] Preferably, generating a conflict-free instruction sequence specifically includes: maintaining a real-time resource topology graph, where node attributes include the currently allocated resources and remaining capacity of each physical node; The resource allocation action sequence is subjected to a four-layer conflict detection: The first layer sums the memory requests of all actions to be executed on the same physical node and compares them with the actual remaining memory of the node to identify memory oversubscription conflicts; the second layer detects whether multiple actions request overlapping processor core number ranges to identify core binding conflicts; the third layer detects whether the resources requested by a single action span two memory nodes to identify cross-node access conflicts; the fourth layer sums the concurrent transmission requests on the same interconnect bus and compares them with the bus bandwidth limit to identify network bandwidth conflicts. Conflict resolution is performed based on task priority scores: high-priority task actions retain their original allocation, while low-priority task actions are postponed to the next scheduling cycle in terms of conflicting resources; after resolution, all actions are sorted according to memory node affinity, and actions belonging to the same memory node domain are arranged in batches and consecutively to generate a conflict-free instruction sequence.

[0014] Preferably, an AI computing power scheduling system for private large-scale models includes: The encoding module performs attention encoding on multi-source heterogeneous data to obtain format recognition results and acquisition priority labels; The query module queries the video memory usage calibration table based on the format recognition results to obtain the video memory increment prediction value, and combines the video memory increment prediction value with the current video memory status to form a video memory pressure pre-transmission signal. The mapping module selects and filters redundant batches based on priority labels to obtain a high-quality data stream; it extracts the type ratio, sparsity density and sequence dimension features of the high-quality data stream and maps them into a computational affinity encoding vector through a graph neural network. The resource allocation module queries the encrypted computing power calibration table based on the format recognition result to obtain the estimated consumption of security processing, declares the security computing power pre-occupancy request to the scheduler and deducts the corresponding resources; constructs a state space with computing power affinity encoding vector, memory pressure pre-transmission signal and resource status, and outputs the resource allocation action sequence. The instruction generation module performs memory oversubscription, processor core binding conflict, NUMA node crossing and network bandwidth conflict detection on the resource allocation action sequence, performs conflict resolution based on task priority scoring and rearranges according to NUMA affinity to generate a conflict-free instruction sequence.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention introduces a "memory pressure pre-transmission mechanism," which can predict the memory resource usage trends of different data types, batch sizes, and feature dimensions before data batches officially enter the training or inference process. Combined with real-time memory status, it forms a memory pressure pre-transmission signal, thereby achieving proactive awareness of resource risks. This invention can avoid memory oversubscription, frequent paging, and task interruption issues during the scheduling phase, reducing the probability of GPU memory fragmentation and improving resource stability and continuous operation capabilities during large model training and inference. It is particularly suitable for resource-constrained, task-intensive computing environments in private deployment scenarios.

[0016] 2. This invention achieves dynamic matching between data structure features and underlying computing resources by constructing a "computing power affinity encoding vector." By extracting type proportions, sparsity density, and sequence dimension features from high-quality data streams, and combining this with graph neural networks to aggregate and model inter-batch structural relationships, the system can accurately identify the compatibility relationships between different data tasks and parallel-intensive computing power, sequence-processing computing power, and vector-accelerated computing power. This invention can adaptively generate resource allocation strategies based on the characteristics of the data itself, improving the utilization rate of heterogeneous computing resources, reducing computing power idleness and inefficient scheduling, thereby improving overall throughput efficiency and task execution performance.

[0017] 3. This invention establishes a multi-level conflict detection and resolution mechanism covering video memory, processor cores, NUMA nodes, and network bandwidth. After generating resource allocation actions, potential resource conflicts are uniformly verified, and dynamic pruning and NUMA affinity rearrangement are implemented based on task priority. This invention can comprehensively consider various underlying hardware constraints, reduce latency losses caused by cross-NUMA access, reduce network congestion and core contention issues, while ensuring the stable execution of high-priority tasks, and improving the overall scheduling reliability and execution efficiency of complex private computing clusters in high-concurrency scenarios. Attached Figure Description

[0018] Figure 1 A schematic diagram of an AI computing power scheduling method for private large-scale models provided by the present invention; Figure 2 A schematic diagram of an AI computing power scheduling system for private large-scale models provided by the present invention; Figure 3 A schematic diagram of the computational affinity coding vector generation process provided by the present invention; Figure 4 This is a schematic diagram of the logic flow of the four-layer conflict detection and resolution mechanism of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0020] The method described in this invention is deployed on a private large-scale model platform of a financial enterprise. The platform's hardware environment includes several GPU computing nodes, each with a fixed capacity of video memory and no elastic expansion capability. Data sources include various heterogeneous formats such as text-based transaction records, image-based vouchers, audio recordings, and structured transaction logs, all originating from different business systems within the enterprise's intranet. The entire method performs format recognition, priority labeling, redundancy filtering, video memory pressure estimation, and computational affinity encoding before the data reaches the GPU. Conflict resolution is completed before the scheduler outputs the resource allocation instruction sequence, ensuring video memory security and efficient utilization of computing power during multi-task concurrency in a private environment.

[0021] Example 1: Please see Figures 1 to 2This invention provides an AI computing power scheduling method for private large-scale models, applied to an AI computing power scheduling system for private large-scale models. The technical solution is as follows: Attention encoding is performed on multi-source heterogeneous data to obtain format recognition results and acquisition priority labels; based on the format recognition results, a memory usage calibration table is queried to obtain the memory increment prediction value, and the memory increment prediction value and the current memory status are combined to form a memory pressure pre-transmission signal; redundant batches are selected and filtered using priority labels to obtain a high-quality data stream after integrity verification, packet loss retransmission processing, and redundant batch filtering; the type ratio and sparsity of the high-quality data stream are extracted. Density and sequence dimension features are mapped into a computing power affinity encoding vector via a graph neural network. Based on the format recognition result, the encrypted computing power calibration table is queried to obtain the estimated consumption of secure processing. Combined with the secure computing power pre-occupancy request, the scheduler is notified and the corresponding resources are deducted. A state space is constructed using the computing power affinity encoding vector, the memory pressure pre-transmission signal, and the resource status, and a resource allocation action sequence is output. The resource allocation action sequence is subjected to memory oversubscription, processor core binding conflict, NUMA node crossing, and network bandwidth conflict detection. Conflict resolution is performed based on task priority scoring, and the sequence is rearranged according to NUMA affinity to generate a conflict-free instruction sequence.

[0022] This embodiment describes the complete execution flow of an AI computing power scheduling method for private large-scale models. The steps are connected sequentially to form a complete scheduling link from data access to conflict-free resource allocation instruction output.

[0023] Step 1: Perform attention encoding on multi-source heterogeneous data to obtain format recognition results and acquisition priority labels; Furthermore, attention encoding is performed on the multi-source heterogeneous data to obtain format recognition results and collection priority labels, specifically including: Field distribution features, encoding format features, and dimensional organization features are extracted from batches of multi-source heterogeneous data and combined to construct a query vector. Attention weight matching calculation is performed between the query vector and the predefined data type standard template vector, and the format recognition result is determined according to the highest matching confidence. The collection priority label is generated based on the actual consumption rate of data types and the backlog of the training queue during the current training phase as dynamic states, and the training improvement ratio as a feedback signal. The collection frequency weight and collection range of data types are adjusted, and the collection priority label is output.

[0024] Specifically, for format recognition, the structure of the current data batch is parsed at the data acquisition layer entry point. Field distribution features are extracted by statistically analyzing the numerical range distribution, missing rate, and inter-field correlation of each data field within the batch, forming a feature sub-vector describing the field distribution pattern. Encoding format features are extracted by detecting the encoding protocol type (e.g., JSON, Protobuf, binary stream, etc.) and its internal structure identifier used in the batch data, forming an encoding format feature sub-vector. Dimensional organization features are extracted by analyzing the number of dimensions, the length distribution of each dimension, and the channel arrangement of the batch data, forming a dimension organization feature sub-vector. These three types of sub-vectors are concatenated to obtain the query vector for the current batch.

[0025] A standard template vector library is established during the system deployment phase: Sufficient samples of various data types, including text, images, audio, video, and structured data, are collected. For each type of sample, the three feature sub-vectors mentioned above are extracted, and their average is calculated to obtain the standard template vector for that type, which is then written to the local template library. During runtime, attention weights are calculated for both the query vector and all template vectors. The weight values ​​reflect the structural similarity between the query vector and each template vector. The data type label corresponding to the template vector with the highest weight is taken as the format recognition result, and this highest weight value is the recognition confidence level. When the highest confidence level is lower than the recognition confidence lower limit threshold, the batch is marked as pending confirmation and enters the manual review queue, not participating in subsequent automatic processing. The recognition confidence lower limit threshold is determined through testing and adjustment on mixed-type datasets during the deployment phase to keep the false recognition rate within an acceptable range.

[0026] Priority labels for data collection are generated dynamically based on three quantifiable technical indicators: data type consumption rate, training queue backlog, and validation metric changes. The following rules apply: the actual consumption rate of each data type during the current training phase is obtained by statistically analyzing the number of batches of each data type consumed by the training task per unit time; the training queue backlog is obtained by real-time statistical analysis of the number of batches of each data type in the waiting queue; and the validation metric change rate is obtained by periodically evaluating the change in the current model's evaluation metric on the validation set relative to historical benchmarks, reflecting the technical contribution of this data type to the training effect. The rules for adjusting collection frequency weights are as follows: when the consumption rate of a certain data type consistently exceeds the supply rate and the corresponding validation metric shows an upward trend, the collection frequency weight for that type is increased by a preset step size, and its data source range is expanded; when the queue backlog of a certain data type consistently exceeds a preset backlog limit, the collection frequency weight for that type is decreased by a preset step size, tilting collection resources towards data types with insufficient supply; when the difference between the consumption rate and supply rate of each data type is within a preset equilibrium range, the current weight remains unchanged. The above rules are executed cyclically with a fixed scheduling cycle. At the end of each cycle, the corresponding collection priority label is output based on the combination of the current consumption rate and the queue backlog of each type: the type whose consumption rate is continuously greater than the supply rate is output with a high priority label; the type whose queue backlog is continuously greater than the upper limit is output with a low priority label; and the other types are output with a medium priority label.

[0027] Step 2: Based on the format recognition results, query the video memory usage calibration table to obtain the predicted video memory increment value and construct the video memory pressure pre-transmission signal; Furthermore, the process of acquiring the pre-transmission signal of video memory pressure is as follows: for the target processor (graphics processor) model, enumerate the combination of each data type and each batch size, actually execute the loading operation of the corresponding data batch, collect the change in video memory usage before and after loading, and establish a video memory usage calibration table with the triplet of data type label, batch sample quantity, and single sample feature dimension scale as index. After the format recognition result is output, the calibration table is queried by data type label, current batch sample quantity, and single sample feature dimension scale to obtain the predicted value of video memory increment; based on the predicted value of video memory increment and the current real-time video memory occupied by the processor, the expected remaining video memory is obtained.

[0028] Specifically, the calibration table creation process is executed when the system is first deployed on the hardware environment of the target GPU model, and is not repeated during subsequent runtime. For all data types supported by the system, several representative batch sample quantity levels and single-sample feature dimension scale levels are enumerated. For each data type and each combination of quantity and dimension, the following calibration operations are performed: First, the current GPU memory usage is collected through the hardware monitoring interface as a baseline value before loading. Then, the corresponding data batch is actually loaded into the GPU memory. After loading, the current GPU memory usage is collected again through the hardware monitoring interface as a post-loading measurement value. The difference between the two collected values ​​is the measured memory increment value for that combination. Using the triple formed by the data type label, batch sample quantity, and single-sample feature dimension scale as an index, the corresponding measured memory increment value is written into the calibration table. Each combination is calibrated several times, and the average value is written to reduce the impact of random errors in a single measurement. After calibration, the calibration table is serialized and stored in a local configuration file.

[0029] During runtime, after format recognition is complete, the calibration table is queried using the identified data type label, the number of samples in the current batch, and the single-sample feature dimension scale. When the query triplet exactly matches a record in the table, the memory increment value of that record is directly read; when the query triplet does not exactly match a record in the table (e.g., the batch sample number falls between two calibrated levels), the predicted value is calculated by proportionally interpolating the calibration values ​​of adjacent levels. This predicted memory increment value is added to the current processor memory usage collected in real time through the hardware monitoring interface to obtain the expected total memory usage after the batch is loaded. The expected total memory usage is then subtracted from the total memory capacity of the graphics processor to calculate the expected remaining memory. The predicted memory increment value and the current memory usage are packaged into a memory pressure pre-transmission signal and sent to the scheduler through the internal shared memory channel. The scheduler compares the expected remaining GPU memory with the GPU memory safety threshold required by the inference task. This safety threshold is set during deployment based on the peak GPU memory requirements of the target inference task, with added safety redundancy, ensuring sufficient GPU memory space for the inference task after the current batch is loaded. The accuracy of the calibration table is maintained through periodic correction: every fixed batch number, the deviation between the predicted and measured values ​​for each combination is calculated. If the deviation of a combination consistently exceeds the tolerance limit, the corresponding entry is updated with the latest measured average, ensuring continuous convergence of prediction accuracy.

[0030] Step 3: Filter redundant batches with priority labels to obtain a high-quality data stream after integrity verification, packet loss retransmission processing and redundant batch filtering; extract structural features and map them into computing power affinity encoding vectors through graph neural networks; Furthermore, the high-quality data stream acquisition process specifically includes: determining the transmission reliability strategy for each data batch based on the collection priority label: high-priority batches are subject to end-to-end integrity verification to ensure transmission integrity; low-priority batches are subject to a lightweight sequence number mechanism for packet loss detection and selective retransmission after transmission; batches exceeding the retransmission limit are downgraded to low-confidence batches and enter the re-collection queue. Calculate the feature similarity between the current data batch and historically collected batches; batches with feature similarity exceeding a preset threshold are marked as redundant batches and skipped from subsequent transmission; the feature library uses a sliding window mechanism to retain the feature vectors of the most recent batches; after dual processing of reliability screening and redundancy filtering, output a high-quality data stream that has undergone integrity verification, packet loss retransmission processing, and redundant batch filtering.

[0031] Specifically, the transmission reliability strategy is implemented based on the collection priority label. For data batches with high priority labels, a connection-oriented reliable transmission protocol is used. After the connection is established, a sliding window verification mechanism is enabled, and the receiving end verifies the integrity of each received data packet to ensure end-to-end data integrity. For data batches with low priority labels, a connectionless transmission protocol is used. A sequence number occupying a fixed number of bytes is appended to the header of each data packet. The receiving end maintains a receiving window, detects positions where the sequence number is discontinuous to identify packet loss, and sends a selective retransmission request to the sending end for lost packets. The retransmission waiting timeout is set during the deployment phase based on the measured round-trip delay in the private LAN plus the allowed queuing delay. If a data batch still does not arrive completely after the maximum number of retransmissions, the batch is downgraded to a low-confidence batch and enters the re-collection queue. It is re-collected from the original data source and does not participate in subsequent processing to ensure that the data entering the downstream process has basic integrity.

[0032] The redundancy filtering mechanism is implemented by calculating feature similarity. For data batches that pass the reliability screening, a pre-trained lightweight feature extraction network is used to extract compact feature vectors. This feature extraction network is trained on representative samples of various data types during system deployment, and its parameter size is controlled to be on the order of negligible inference latency. After extracting the feature vector of the current batch, cosine similarity is calculated one by one with the feature vectors of the most recent batches retained in the historical feature database. The cosine similarity value ranges from zero to one, and the higher the value, the more similar the semantic content of the two batches of data. When the cosine similarity between the current batch and any record in the historical feature database exceeds a preset filtering threshold, the batch is determined to be a redundant batch and its subsequent transmission is skipped; batches that do not exceed the filtering threshold are determined to be valid batches, their feature vectors are written into the historical feature database, and a sliding window update is triggered—the earliest written historical feature vector is removed from the database, keeping the size of the feature database within the capacity limit determined at the time of deployment.

[0033] The filtering threshold is obtained as follows: During the system deployment phase, several pairs of data batches known to be semantically equivalent (i.e., carrying the same or redundant data content) and several pairs of data batches known to be semantically different are collected. The cosine similarity between each pair is calculated, and the similarity distribution is plotted. The boundary point that minimizes the sum of the misclassification rates of the two distributions (equivalent and non-equivalent batches) is selected as the filtering threshold and stored in the system configuration file. This threshold represents the optimal boundary for semantic equivalence determination. When the data distribution changes significantly, the threshold can be updated by re-executing the above calibration process.

[0034] Furthermore, referring to Figure 3 The process of generating computing power affinity coding vectors specifically includes: sampling batches of data to be processed from the buffer queue of high-quality data streams; extracting three structural features for each data batch: the proportion of data type composition, the ratio of non-zero feature elements in the batch to all feature elements, and the maximum sequence length of samples in the batch; and constructing feature combinations by combining historical data with similar structural features with measured throughput efficiency benchmarks on various computing power resources. A graph structure is constructed using the batches of data to be processed in the buffer queue as nodes and the structural feature similarity between batches as edge weights. If the number of batches to be processed in the current buffer queue is insufficient to form effective neighbor relationships, feature records of historically processed batches are introduced as auxiliary reference nodes. Each node in the graph structure aggregates neighbor feature information based on edge weights, and outputs a fixed-dimensional computing power affinity encoding vector after multi-layer feature transformation. Each component in the computing power affinity encoding vector explicitly quantifies the expected throughput acceleration benefits of the current data stream for parallel intensive computing power, sequence processing computing power, and vector acceleration computing power.

[0035] Specifically, after the high-quality data stream enters the transmission buffer queue, the semantic matching encoder samples the batches of data to be processed in the buffer queue according to a fixed scheduling period. For each sampled data batch, three structural features are extracted: The first is the data type composition ratio, which is the proportion of each data type sample in the batch divided by the total number of samples in the batch. This feature reflects the data composition of the batch and is used to characterize the batch's basic requirement candidate degree for different computing power types. The second is the sparsity density, which is the ratio of the number of non-zero elements in all feature elements in the batch to the total number of feature elements. This feature, combined with the measured throughput efficiency of similar batches in history on various computing power types, reflects the batch's candidate adaptation degree for different computing power types. The third is the sequence dimension, which is the length of the longest sequence among all samples in the batch. This feature, also combined with the historical measured throughput efficiency benchmark, reflects the candidate processing efficiency of the current batch on different computing power types. The final adaptation relationship is determined by the historical measured throughput efficiency benchmark and the graph neural network output, rather than by a single feature.

[0036] Combining the above three structural features, the historical measured throughput efficiency of data batches with similar structural feature combinations (same or similar data types, similar sparse density ranges, similar sequence dimension ranges) on various computing resources is retrieved from the historical benchmark database of scheduling execution feedback. A complete feature combination containing the current structural features and the corresponding historical efficiency benchmark is constructed as the initial features of the graph neural network nodes.

[0037] The graph structure is constructed as follows: using all batches to be processed in the buffer queue as graph nodes, the similarity between the feature vectors of any two nodes is calculated, and an edge is established between the two nodes using this similarity as the edge weight. When the number of batches to be processed in the buffer queue is insufficient to form at least one effective edge between nodes (i.e., the number of batches in the queue is less than two, and node pairs cannot be formed), the feature records of previously processed batches are introduced as auxiliary reference nodes to participate in the graph structure construction. The auxiliary nodes do not output encoding results, but only provide neighbor features for the current node to aggregate, ensuring that the graph neural network has sufficient neighbor information for effective feature propagation.

[0038] The graph neural network employs a two-layer graph sampling and aggregation architecture. Each layer performs the following operations: for each node, its neighboring nodes are sampled weighted by edge weights; the feature vectors of the neighboring nodes are aggregated weighted by edge weights to form a neighbor representation; the neighbor representation is concatenated with the node's own feature vector, and then processed by a linear transformation and activation function to output the updated feature vector of that node in the current layer. After two layers of processing, a fixed-dimensional computational affinity encoding vector is output for each node. This vector contains three components: the first component represents the expected throughput acceleration gain of the current data stream on parallel intensive computing power; the second component represents the expected throughput acceleration gain on sequence processing computing power; and the third component represents the expected throughput acceleration gain on vector acceleration computing power. Higher values ​​for each component indicate that the type of computing power is more efficient in processing the current data stream. The weight parameters of the graph neural network are obtained offline through training on historical scheduling execution logs during the system deployment phase. The training objective is to ensure that each component of the encoding vector is consistent with the measured throughput improvement ratio of the corresponding computing power type. These parameters are written into the model parameter file and are used permanently during runtime without further updates.

[0039] Step 4: Query the encryption computing power calibration table, declare the pre-allocation of secure computing power; construct the state space, and output the resource allocation action sequence; Furthermore, constructing the state space output resource allocation action sequence specifically includes: Benchmark tests of encryption operations were performed on the target processor model for each data type. The processor usage and processing time under each data type and data volume combination, as well as the processor overhead of secure handshake under different concurrent connection numbers, were collected. An encryption computing power calibration table was established with data type and data volume as indexes and encryption processor overhead as values. After the format recognition result is output, the encryption computing power calibration table is queried to obtain the estimated processor resource consumption for the current batch of security processing, and a security computing power pre-occupancy request is sent to the scheduler; the scheduler marks the corresponding processor resource as a security pre-occupancy status in the resource ledger and does not allocate the corresponding resource to the business task within the estimated duration of the batch security processing. The actual available resource status after security pre-allocation, the computing power affinity encoding vector, and the pre-transmission signal of video memory pressure are concatenated into a state vector; the task queue is dynamically sorted in a fine-grained manner according to three dimensions: business importance, time urgency, and resource demand scale, to form a task priority score; with the state vector as input, resource allocation actions are generated for the tasks to be scheduled, including the target computing power node, the amount of video memory requested, the number of processor cores requested, the target memory node number, and the amount of network bandwidth requested. All actions constitute a resource allocation action sequence.

[0040] Specifically, the encryption computing power calibration table is established when the system is first deployed on the target general-purpose processor model's hardware environment. For all data types and data volume levels supported by the system, encryption operations (including symmetric encryption operations and secure handshake processes) of the corresponding type and data volume are actually executed on the target general-purpose processor, and the general-purpose processor usage and complete processing time are collected simultaneously; at the same time, the average general-purpose processor overhead for secure handshakes is collected for different concurrent connection levels. Using the combination of data type and data volume as the index and the encryption general-purpose processor overhead (the product of general-purpose processor usage and processing time, in units of general-purpose processor cores multiplied by seconds) as the value, the encryption computing power calibration table is established and serialized and stored in the local configuration file.

[0041] During runtime, after format recognition, the encrypted computing power calibration table is queried using the data type label and the data volume of the current batch. The corresponding estimated general-purpose processor resource consumption is read and sent to the scheduler via an internal interface in the format of a secure computing power pre-allocation request message. The scheduler marks the corresponding general-purpose processor resource as securely pre-allocated in the resource ledger, freezes this resource for the estimated processing time of the current batch, and does not include it in the total allocable general-purpose processor resource volume, ensuring that the amount of available general-purpose processor resources perceived by the scheduler is consistent with the amount of general-purpose processor resources actually available for the business task. After the security protection module completes the encryption operation, it sends a release signal to the scheduler. The scheduler removes the pre-allocation mark and re-includes the relevant general-purpose processor resources in the allocable resource pool. The accuracy of the calibration table is maintained through periodic deviation statistical correction. If the difference between the actual consumption and the estimated value for several consecutive batches under a certain data type and data volume combination exceeds the tolerance ratio, the corresponding table entry is updated with the latest measured average.

[0042] The state space is constructed as follows: the real available resource state vector after deducting security pre-occupancy, the computing power affinity encoding vector, and the memory pressure pre-transmission signal are concatenated in field order to form a complete state vector, which serves as the input to the scheduling strategy model. Specifically, the real available resource state vector consists of the current remaining memory, remaining general-purpose processor cores, current memory node load, and remaining network bandwidth for each computing node; the computing power affinity encoding vector is the three-component vector output from step three; and the memory pressure pre-transmission signal is a combination of the predicted increment output from step two and the current occupancy.

[0043] Task priority scores are calculated comprehensively from three dimensions: Business importance, determined by the pre-defined weights of the task's business category in the deployment configuration, reflecting its priority at the business level; Time urgency, determined by the ratio of the task's remaining deadline to its historical average execution time (a smaller ratio indicates greater urgency); and Resource requirement, determined by the proportion of the total resources requested by the task relative to the system's current available resources (tasks with larger request ratios show their resource consumption impact in the score). The weights of these three dimensions are configured according to the business scenario during the deployment phase, and a comprehensive weighted average is used to obtain the priority score for each task. The task queue is then sorted in descending order based on these scores to form a task priority sequence.

[0044] The scheduling strategy model takes a complete state vector as input and generates resource allocation actions for each task to be scheduled, according to the priority sequence. Each action contains five parameters: target computing node number, specifying the physical computing node on which the task should be executed; requested GPU memory, specifying the number of bytes of GPU memory required for the task; requested general-purpose processor cores, specifying the number of general-purpose processor cores allocated to the task; target memory node number, specifying the memory node accessed by the task, corresponding to the NUMA domain of the target computing node; and requested network bandwidth, specifying the network bandwidth reserved for data transmission by the task. All task actions are arranged sequentially to form a resource allocation action sequence, which is output to the conflict resolution engine.

[0045] Step 5: Perform conflict detection and generate a conflict-free instruction sequence; Furthermore, referring to Figure 4 Generate a conflict-free instruction sequence, specifically including: maintaining a real-time resource topology graph, where node attributes include the currently allocated resources and remaining capacity of each physical node; The resource allocation action sequence is subjected to a four-layer conflict detection: The first layer sums the memory requests of all actions to be executed on the same physical node and compares them with the actual remaining memory of the node to identify memory oversubscription conflicts; the second layer detects whether multiple actions request overlapping processor core number ranges to identify core binding conflicts; the third layer detects whether the resources requested by a single action span two memory nodes to identify cross-node access conflicts; the fourth layer sums the concurrent transmission requests on the same interconnect bus and compares them with the bus bandwidth limit to identify network bandwidth conflicts. Conflict resolution is performed based on task priority scores: high-priority task actions retain their original allocation, while low-priority task actions are postponed to the next scheduling cycle in terms of conflicting resources; after resolution, all actions are sorted according to memory node affinity, and actions belonging to the same memory node domain are arranged in batches and consecutively to generate a conflict-free instruction sequence.

[0046] Specifically, the real-time resource topology graph is maintained by the conflict resolution engine. In the topology graph, each physical computing node corresponds to a graph node. Node attributes contain two types of data: currently allocated resources, which is the total amount of resources occupied by historical actions that have been confirmed and executed within the current scheduling cycle; and remaining capacity, which is the difference between the total hardware capacity of the physical node and the currently allocated resources. The virtualization execution layer reports the actual resource usage of each node to the conflict resolution engine at fixed time intervals. Upon receiving the report, the conflict resolution engine refreshes the attribute values ​​of the corresponding nodes in the topology graph to ensure that the topology graph reflects the current true state of the system.

[0047] The four-layer conflict detection is executed sequentially. After each layer of detection, the identified conflicting actions are marked but not processed immediately; they are resolved uniformly after all four layers of detection are completed. The first layer is memory oversubscription conflict detection: For all actions in the current action sequence that point to the same physical node, the memory requests of each action are accumulated to obtain the total memory requests for that node in the current scheduling cycle. This is compared with the remaining memory capacity of that node in the topology graph. If the total requested memory exceeds the remaining capacity, the lowest priority actions with the smallest requested memory on that node are marked as oversubscription conflicting actions, until the remaining total requested memory does not exceed the remaining capacity. The second layer is processor core binding conflict detection: For actions in the current action sequence that request processor core binding, each pair of actions is checked for numerical overlap in the range of processor core numbers requested (i.e., the same core number exists in two ranges). Among the overlapping action pairs, the action with lower priority is marked as a core binding conflicting action. Layer 3 NUMA Node Cross-Border Conflict Detection: For each action in the current action sequence, check whether the NUMA domain corresponding to the target computing node number and the NUMA domain corresponding to the target memory node number are the same NUMA domain. If they are not in the same NUMA domain (i.e., the computing resources and memory resources requested by the action cross the NUMA node boundary), the action is marked as a cross-node access conflict action. Such crossings will cause a significant increase in memory access latency. Layer 4 Network Bandwidth Conflict Detection: For all actions in the current action sequence that share the same interconnect bus (including PCIe bus or high-speed interconnect bus), the network bandwidth requested by each action is accumulated and compared with the physical bandwidth limit of the bus. If the total requested amount exceeds the limit, the few low-priority actions with the smallest requested bandwidth on the bus are marked as bandwidth conflict actions, until the remaining total requested amount does not exceed the bandwidth limit.

[0048] Conflict resolution employs a retention or deferral strategy based on task priority scores. Among actions marked as conflicting, the action with the highest priority score retains its original request amount for the current scheduling cycle; conflicting actions with lower priority scores are deferred entirely to the next scheduling cycle, meaning they will re-compete for resources at the start of the next cycle with their then-current resource status. The next scheduling cycle is defined as the time elapsed after a fixed-length scheduling cycle from the start of the current cycle. The scheduling cycle length is determined during deployment based on system task response latency requirements. The resolution process does not partially reduce the request amount for any action to avoid resource fragmentation. Each action is either executed in the current cycle with its original request amount or deferred entirely, ensuring deterministic resource allocation. To prevent low-priority tasks from being delayed due to the continuous arrival of high-priority tasks, the system maintains a continuous deferral counter for each deferred action: whenever an action is deferred entirely due to conflict in a scheduling cycle, its continuous deferral count is incremented; if the action is successfully executed in a scheduling cycle, the continuous deferral count is reset to zero. During the conflict resolution phase of each scheduling cycle, the number of consecutive postponements of each action is multiplied by a preset aging step size coefficient and then added to its original priority score to obtain the aging-corrected priority score for that action in the current cycle. The aging step size coefficient is determined during the deployment phase based on the business scenario's requirement for the maximum acceptable number of postponements, ensuring that actions whose consecutive postponements reach the preset maximum postponement limit have an aging-corrected priority score that is not lower than the highest value of the original priority scores of all actions in the current scheduling cycle. During conflict resolution, the aging-corrected priority score replaces the original priority score as the criterion for retention or postponement, guaranteeing that after each action has been postponed for no more than the maximum postponement limit for a given number of scheduling cycles, its aging-corrected score will inevitably be higher than other lower-priority actions in the current conflict, thus allowing it to be executed when resources permit and preventing long-term starvation.

[0049] After resolution, all retained actions within the current cycle are sorted according to NUMA affinity: actions belonging to the same NUMA domain (i.e., the target computing node and the target memory node both belong to the same NUMA node action group) are batched and arranged consecutively to ensure that all resource allocation operations within the same NUMA domain are executed continuously at the virtualization execution layer, reducing context switching overhead across NUMA domains and lowering memory access latency. Action groups from different NUMA domains are arranged in ascending order of allocated resource amounts, prioritizing the filling of NUMA domains with lower resource usage to balance the load across domains. After sorting, the action sequence is converted into a conflict-free resource allocation instruction sequence that can be parsed by the virtualization execution layer and executed sequentially by the virtualization execution layer.

[0050] Example 2: This embodiment, based on Embodiment 1, focuses on describing the specific implementation process of three extended mechanisms: heat-aware computing power scheduling, memory bandwidth saturation awareness, and predictive pre-migration. The remaining parts of the system, including attention encoding and format recognition of multi-source heterogeneous data, acquisition of memory pressure pre-transmission signals, reliability screening and redundancy filtering of high-quality data streams, construction of the basic graph neural network for computing power affinity encoding vectors, encryption computing power calibration and state vector concatenation, and four-layer conflict detection and NUMA affinity sorting, are all the same as in Embodiment 1 and will not be repeated here.

[0051] Furthermore, the current value of the junction temperature sensor of each computing node processor is read at a preset sampling period, and the difference between the rated maximum junction temperature of the processor model and the current junction temperature is recorded as the thermal margin value; at the same time, the measured operating frequency of the current processor is read, and the ratio of the measured operating frequency to the nominal base frequency of the processor is recorded as the frequency retention rate; if the thermal margin value is lower than the preset thermal warning threshold, or the frequency retention rate is lower than the preset frequency reduction trigger threshold, the corresponding node is marked as thermally limited; when constructing the state vector, the thermal margin value and the frequency retention rate are added to the state vector as independent feature dimensions; when the scheduler generates resource allocation actions for thermally limited nodes, the upper limit of the number of processor cores that can be allocated to the node is multiplied by the frequency retention rate and rounded down, and the reduced number of cores is used as the upper limit of the number of cores that can actually be allocated to the node, so as to avoid over-allocating tasks to nodes that are continuously reducing their frequency.

[0052] Specifically, the scheduling daemon periodically reads the junction temperature sensor values ​​of the processors on each compute node in the cluster via the processor management bus interface, using a 500-millisecond sampling period. Simultaneously, it reads the current measured core operating frequency of the processor via the hardware performance counter interface. The thermal margin value is obtained by subtracting the currently read junction temperature from the rated maximum junction temperature specified in the processor model's datasheet. The frequency hold-up rate is obtained by dividing the current measured frequency by the nominal base frequency of the processor model. When the measured frequency is higher than the nominal base frequency (e.g., when a general-purpose processor is in Turbo Boost mode), the frequency hold-up rate is truncated by an upper limit of 1 to avoid overflow during the calculation.

[0053] The preset thermal warning threshold is determined as follows: A continuous stress test is performed on the target processor model, and the thermal margin value corresponding to the processor's operation from full load to triggering the first active frequency reduction is recorded. A multiple of this critical value is used as the safety margin coefficient, and the amplified result is set as the thermal warning threshold to ensure that the scheduling strategy switch is completed before frequency reduction occurs. The preset frequency reduction trigger threshold is determined as follows: An increasing load is gradually applied to the target processor, and the actual computing throughput corresponding to each gradient is recorded. The frequency retention rate corresponding to the actual throughput decreasing more than 3% relative to the nominal value is taken as the trigger threshold, which is 0.97 in this embodiment. The specific value depends on the thermal design power parameters of the processor model.

[0054] When a computing node's heat margin value falls below the heat warning threshold or its frequency maintenance rate falls below the frequency reduction trigger threshold, the node is marked as thermally constrained. When the scheduler generates a resource allocation action for a thermally constrained node, it multiplies the node's original allocatable core count by the frequency maintenance rate and rounds down. The resulting integer is written into the resource allocation action as the upper limit for the node's core allocation in the current scheduling cycle. The heat margin value and the frequency maintenance rate are simultaneously added as independent dimensions to the state vector for the policy network to comprehensively consider in subsequent scheduling decisions.

[0055] Furthermore, when extracting the computing power affinity coding vector, memory bandwidth saturation features are also extracted for each NUMA node: the total number of actual read and write bytes of the memory controller of each NUMA node is accumulated within a preset sampling window, and this total number of bytes is divided by the product of the nominal peak bandwidth of the memory controller of the node and the sampling window duration to obtain the memory bandwidth utilization rate of the node; the mean and fluctuation amplitude of the bandwidth utilization rate sequence of each node in multiple consecutive sampling periods are used to construct a bandwidth saturation statistic; if the mean bandwidth utilization rate of a node continuously exceeds the preset high bandwidth load threshold and the fluctuation amplitude is lower than the preset stable bandwidth threshold, the node is determined to be in a continuous bandwidth saturation state; the bandwidth saturation statistic of each NUMA node is used as an additional dimension to be concatenated to the computing power affinity coding vector output by the graph neural network, so that the scheduler can reduce the allocation weight of memory-intensive tasks of bandwidth-saturated nodes when generating resource allocation actions.

[0056] Specifically, during the feature extraction phase of the graph neural network, the system additionally collects the memory bandwidth utilization rate of each NUMA node. The sampling window is set to 100 milliseconds. Within the window period, the total number of bytes actually read and written to the memory of the node is accumulated and recorded through the memory controller performance counter. This is then divided by the product of the nominal peak bandwidth of the memory controller of the node and the sampling window duration to obtain the bandwidth utilization rate between 0 and 1.

[0057] The bandwidth saturation statistic is calculated by taking the bandwidth utilization rate sequence of each node over 10 consecutive sampling periods and calculating the difference between its mean, maximum, and minimum values ​​(as the fluctuation amplitude). The preset high-load bandwidth threshold is determined as follows: The concurrency of memory-intensive tasks is gradually increased on the target node. The growth factor of the node's memory access latency relative to idle latency is recorded when the mean bandwidth utilization rate reaches each gradient. The mean bandwidth utilization rate corresponding to a memory access latency increase exceeding 20% ​​is taken as the high-load bandwidth threshold, which is 0.82 in this embodiment. The specific value depends on the memory model and number of channels. The preset bandwidth stability threshold is set to 0.06, indicating that when the bandwidth utilization rate fluctuation amplitude is less than 6% within a continuous sampling window, the current bandwidth load state is considered a continuously stable high load, rather than an instantaneous peak.

[0058] The bandwidth saturation statistics of each NUMA node (two values: mean and fluctuation amplitude) are concatenated into an additional feature dimension, which is added to the feature vector of each batch of nodes in the graph neural network. Finally, it is input into the graph neural network along with the batch graph structure to participate in neighbor feature aggregation and multi-layer feature transformation, so that the output computing power affinity encoding vector contains the encoding information of the memory bandwidth carrying capacity of the corresponding node.

[0059] Furthermore, the generation of conflict-free instruction sequences also includes a predictive pre-migration step: maintaining historical memory usage sequences, processor core usage sequences, and network bandwidth usage sequences for each computing node in scheduling cycles; calculating smoothed estimates for each resource dimension using an exponentially weighted moving average method for each of the above resource sequences, and using these smoothed estimates as the predicted values ​​for the next scheduling cycle. This prediction method is suitable for scenarios where resource usage does not show a significant monotonic trend in the short term; the smoothing coefficient is adaptively adjusted based on the absolute mean of the difference between historical measured values ​​and previous predicted values: when the recent absolute error mean exceeds a preset high-sensitivity error threshold, the smoothing coefficient is increased to enhance... To track recent changes, the smoothing coefficient is reduced when the error is below a preset low-sensitivity threshold to maintain stable predictions. If the predicted memory usage of a node exceeds the product of the actual memory capacity of the node and the preset pre-migrating memory threshold, or if the predicted number of processor cores exceeds the product of the total number of cores of the node and the preset pre-migrating core threshold, a pre-migrating instruction is appended to the end of the conflict-free instruction sequence in the current scheduling cycle. Subsequent batches of tasks with priority scores lower than the preset migration trigger priority threshold are scheduled to standby nodes with sufficient resource reserves. The allocated resource amount of relevant nodes in the real-time resource topology graph is updated synchronously to prevent emergency interrupt migration from being triggered due to resource exhaustion in the next scheduling cycle.

[0060] After completing four-layer conflict detection and generating a conflict-free instruction sequence, the instruction generation module performs predictive pre-migration judgment. The system maintains historical memory usage sequences, processor core usage sequences, and network bandwidth usage sequences for each computing node, each with a length of 20 scheduling cycles, and stores them in a circular buffer.

[0061] For the three types of sequences mentioned above, a smoothed estimate of the current moment is calculated using an exponentially weighted moving average, which is then used as the predicted value for the next scheduling cycle. This prediction method is suitable for scenarios where resource occupancy does not show a significant monotonic trend in the short term. The initial value of the smoothing coefficient is set to 0.3. At the end of each scheduling cycle, the absolute value of the difference between the measured value of the current cycle and the smoothed estimate of the previous cycle is calculated as the absolute error of the current cycle. The average absolute error of the last 5 scheduling cycles is used as the representative value of the recent error. The preset high-sensitivity and low-sensitivity error thresholds are determined as follows: The resource sequences collected by the target cluster under a typical training task for 30 minutes are statistically analyzed. The 75th percentile of the absolute error sequence is taken as the high-sensitivity error threshold, and the 25th percentile of the absolute error sequence is taken as the low-sensitivity error threshold. When the representative value of the recent error is higher than the high-sensitivity error threshold, the smoothing coefficient is increased by 0.05 up to the upper limit of 0.8; when it is lower than the low-sensitivity error threshold, the smoothing coefficient is decreased by 0.05 down to the lower limit of 0.1.

[0062] The preset pre-migration memory threshold is set to 0.85, meaning pre-migration is triggered when the predicted memory usage of a node exceeds 85% of its memory capacity. The preset pre-migration core threshold is set to 0.88, meaning pre-migration is triggered when the predicted processor core usage exceeds 88% of the node's total cores. The calibration method for these two thresholds is as follows: multiple typical training tasks are run on the target cluster, and the number of remaining scheduling cycles between the resource usage reaching each candidate threshold value and the actual resource exhaustion is counted. The candidate value that provides at least two scheduling cycles for migration operations is selected as the final threshold. The preset migration trigger priority threshold is the median of the priority score sequence of all tasks in the current task queue. Tasks below the median are selected as migration candidates, and their subsequent batches are scheduled to standby nodes with sufficient resource reserves. The allocated resource amounts of relevant nodes in the real-time resource topology graph are updated synchronously to prevent emergency migration interruption due to resource exhaustion in the next scheduling cycle.

[0063] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for scheduling AI computing power for private large-scale models, characterized in that, include: Attention encoding is performed on multi-source heterogeneous data to obtain format recognition results and collection priority labels; Based on the format recognition results, the video memory usage calibration table is queried to obtain the video memory increment prediction value. The video memory increment prediction value and the current video memory status are combined to form a video memory pressure pre-transmission signal. Redundant batches are selected and filtered by priority labels to obtain high-quality data streams; the type ratio, sparsity density and sequence dimension features of the high-quality data streams are extracted and mapped into computing power affinity encoding vectors via graph neural networks; Based on the format recognition result, the encrypted computing power calibration table is queried to obtain the estimated consumption of security processing. Combined with the security computing power pre-occupancy request, the scheduler is declared and the corresponding resources are deducted. The state space is constructed with the computing power affinity encoding vector, the memory pressure pre-transmission signal and the resource status, and the resource allocation action sequence is output. The system performs memory oversubscription, processor core binding conflict, NUMA node crossing, and network bandwidth conflict detection on the resource allocation action sequence. It then resolves conflicts based on task priority scores and rearranges them according to NUMA affinity to generate a conflict-free instruction sequence.

2. The AI ​​computing power scheduling method for private large-scale models according to claim 1, characterized in that: Attention encoding is performed on multi-source heterogeneous data to obtain format recognition results and acquisition priority labels, specifically including: Field distribution features, encoding format features, and dimensional organization features are extracted from batches of multi-source heterogeneous data and combined to construct a query vector. Attention weight matching calculation is performed between the query vector and the predefined data type standard template vector, and the format recognition result is determined according to the highest matching confidence. The collection priority label is generated based on the actual consumption rate of data types and the backlog of the training queue during the current training phase as dynamic states, and the training improvement ratio as a feedback signal. The collection frequency weight and collection range of data types are adjusted, and the collection priority label is output.

3. The AI ​​computing power scheduling method for private large-scale models according to claim 1, characterized in that: The process of acquiring the pre-transmission signal of video memory pressure is as follows: For the target processor model, enumerate the combination of each data type and each batch size, actually execute the loading operation of the corresponding data batch, collect the change in video memory usage before and after loading, and establish a video memory usage calibration table with the triplet of data type label, batch sample quantity, and single sample feature dimension scale as index. After the format recognition result is output, the calibration table is queried by data type label, current batch sample quantity, and single sample feature dimension scale to obtain the predicted value of video memory increment; based on the predicted value of video memory increment and the current real-time video memory occupied by the processor, the expected remaining video memory is obtained.

4. The AI ​​computing power scheduling method for private large-scale models according to claim 1, characterized in that: The high-quality data stream acquisition process specifically includes: determining the transmission reliability strategy for each data batch based on the collection priority label: high-priority batches are subject to end-to-end integrity verification to ensure transmission integrity; low-priority batches are subject to a lightweight sequence number mechanism for packet loss detection and selective retransmission after transmission; batches that exceed the retransmission limit are downgraded to low-confidence batches and enter the re-collection queue. Calculate the feature similarity between the current data batch and historically collected batches; batches with feature similarity exceeding a preset threshold are marked as redundant batches and skipped from subsequent transmission; the feature library uses a sliding window mechanism to retain the feature vectors of the most recent batches; after dual processing of reliability screening and redundancy filtering, a high-quality data stream is output.

5. The AI ​​computing power scheduling method for private large-scale models according to claim 1, characterized in that: The specific steps for generating the computing power affinity coding vector include: sampling batches of data to be processed from the buffer queue of high-quality data streams; extracting three structural features for each data batch: the proportion of data type composition, the ratio of non-zero feature elements in the batch to all feature elements, and the maximum sequence length of samples in the batch; and constructing feature combinations by combining historical data with similar structural features with measured throughput efficiency benchmarks on various computing power resources. A graph structure is constructed using the batches of data to be processed in the buffer queue as nodes and the structural feature similarity between batches as edge weights. If the number of batches to be processed in the current buffer queue is insufficient to form effective neighbor relationships, feature records of historically processed batches are introduced as auxiliary reference nodes. Each node in the graph structure aggregates neighbor feature information based on edge weights, and outputs a fixed-dimensional computing power affinity encoding vector after multi-layer feature transformation. Each component in the computing power affinity encoding vector explicitly quantifies the expected throughput acceleration benefits of the current data stream for parallel intensive computing power, sequence processing computing power, and vector acceleration computing power.

6. The AI ​​computing power scheduling method for private large-scale models according to claim 1, characterized in that: The specific steps involved in constructing the state space output resource allocation action sequence are as follows: Benchmark tests of encryption operations were performed on the target processor model for each data type. The processor usage and processing time under each data type and data volume combination, as well as the processor overhead of secure handshake under different concurrent connection numbers, were collected. An encryption computing power calibration table was established with data type and data volume as indexes and encryption processor overhead as values. After the format recognition result is output, the encryption computing power calibration table is queried to obtain the estimated processor resource consumption for the current batch of security processing, and a security computing power pre-occupancy request is sent to the scheduler; the scheduler marks the corresponding processor resource as a security pre-occupancy status in the resource ledger and does not allocate the corresponding resource to the business task within the estimated duration of the batch security processing. The actual available resource status after security pre-allocation, the computing power affinity encoding vector, and the pre-transmission signal of video memory pressure are concatenated into a state vector; the task queue is dynamically sorted in a fine-grained manner according to three dimensions: business importance, time urgency, and resource demand scale, to form a task priority score; with the state vector as input, resource allocation actions are generated for the tasks to be scheduled, including the target computing power node, the amount of video memory requested, the number of processor cores requested, the target memory node number, and the amount of network bandwidth requested. All actions constitute a resource allocation action sequence.

7. The AI ​​computing power scheduling method for private large-scale models according to claim 1, characterized in that: Generate a conflict-free instruction sequence, specifically including: maintaining a real-time resource topology graph, where node attributes include the currently allocated resources and remaining capacity of each physical node; The resource allocation action sequence is subjected to a four-layer conflict detection: The first layer sums the memory requests of all actions to be executed on the same physical node and compares them with the actual remaining memory of the node to identify memory oversubscription conflicts; the second layer detects whether multiple actions request overlapping processor core number ranges to identify core binding conflicts; the third layer detects whether the resources requested by a single action span two memory nodes to identify cross-node access conflicts; the fourth layer sums the concurrent transmission requests on the same interconnect bus and compares them with the bus bandwidth limit to identify network bandwidth conflicts. Conflict resolution is performed based on task priority scores: high-priority task actions retain their original allocation, while low-priority task actions are postponed to the next scheduling cycle in terms of conflicting resources; after resolution, all actions are sorted according to memory node affinity, and actions belonging to the same memory node domain are arranged in batches and consecutively to generate a conflict-free instruction sequence.

8. An AI computing power scheduling system for private large-scale models, characterized in that, include: The encoding module performs attention encoding on multi-source heterogeneous data to obtain format recognition results and acquisition priority labels; The query module queries the video memory usage calibration table based on the format recognition results to obtain the video memory increment prediction value, and combines the video memory increment prediction value with the current video memory status to form a video memory pressure pre-transmission signal. The mapping module selects and filters redundant batches based on priority labels to obtain a high-quality data stream; it extracts the type ratio, sparsity density and sequence dimension features of the high-quality data stream and maps them into a computational affinity encoding vector through a graph neural network. The resource allocation module queries the encrypted computing power calibration table based on the format recognition result to obtain the estimated consumption of security processing, declares the security computing power pre-occupancy request to the scheduler and deducts the corresponding resources; constructs a state space with computing power affinity encoding vector, memory pressure pre-transmission signal and resource status, and outputs the resource allocation action sequence. The instruction generation module performs memory oversubscription, processor core binding conflict, NUMA node crossing and network bandwidth conflict detection on the resource allocation action sequence, performs conflict resolution based on task priority scoring and rearranges according to NUMA affinity to generate a conflict-free instruction sequence.

Images