Resource scheduling method and device, readable storage medium and computer program product

Abstract

The application relates to the technical field of data processing, in particular to a resource scheduling method and device, a readable storage medium and a computer program product. The target object with high frequency access is allocated to a low-delay node, and the target object with low frequency access is allocated to a high-delay node, so that the core service or the task sensitive to access delay can obtain low-delay storage resources, the overall resource utilization of the memory pool is improved, the negative influence caused by the access delay is reduced while the low-cost advantage of the high-delay memory node is reserved, and the balance between the storage performance and the cost is realized.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a resource scheduling method, apparatus, readable storage medium, and computer program product. Background Technology

[0002] The Compute Express Link (CXL) memory pool, through its cache-coherent CXL protocol, enables memory resource pooling across multiple servers, allowing compute nodes to acquire memory from the pool on demand. Currently, in the heterogeneous media convergence CXL memory pool, besides CXL Dynamic Random Access Memory (DRAM) devices, CXL Hybrid Solid State Drives (SSDs) are a key carrier for achieving memory and storage tier convergence. Their core storage medium is flash memory, which has a lower unit storage cost compared to DRAM cache media.

[0003] While CXL hybrid SSDs reduce the storage cost of CXL memory pools, they also introduce higher access latency, which can lead to high and fluctuating memory access latency on compute nodes, making it difficult to fully leverage the resource utilization efficiency and cost advantages of the pooled architecture. Summary of the Invention

[0004] This application provides a resource scheduling method, apparatus, readable storage medium, and computer program product to at least solve the problem of low resource utilization efficiency in pooled architectures in related technologies.

[0005] This application provides a resource scheduling method, including: Obtain multidimensional attribute data corresponding to multiple candidate nodes, including at least one compute node and multiple memory nodes; Based on the resource requirement information and multi-dimensional attribute data of the target object, the target node corresponding to the target object is determined from multiple candidate nodes; wherein, if the access frequency of the target object is greater than a first frequency threshold, the target node corresponding to the target object is determined from a first type of memory node among at least one computing node and multiple memory nodes; if the access frequency of the target object is less than or equal to the first frequency threshold, the target node corresponding to the target object is determined from a second type of memory node among multiple memory nodes, wherein the access latency of the first type of memory node is less than the access latency of the second type of memory node; Schedule the target object to the target node.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described resource scheduling methods when executing the computer program.

[0007] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described resource scheduling methods.

[0008] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described resource scheduling methods.

[0009] This application assigns frequently accessed target objects to low-latency nodes and infrequently accessed target objects to high-latency nodes, enabling core business operations or tasks sensitive to access latency to obtain low-latency storage resources, improving the overall resource utilization of the memory pool, and reducing the negative impact of access latency while retaining the low-cost advantage of high-latency memory nodes, thus achieving a balance between storage performance and cost. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A schematic diagram of a CXL cluster provided in an embodiment of this application; Figure 2 A flowchart of a resource scheduling method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a CXL memory pool awareness system provided in an embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0013] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0014] In traditional server architectures, memory is tightly coupled with the Central Processing Unit (CPU). This rigid configuration model leads to performance bottlenecks due to insufficient memory on individual servers in the face of the explosive growth in memory capacity and bandwidth demands from applications such as Artificial Intelligence (AI) and Big Data. On the other hand, the vast majority of memory is idle during off-peak hours, resulting in huge resource waste and persistently high total cost of ownership.

[0015] The CXL memory pool was built precisely to resolve this fundamental contradiction. Through the CXL protocol, which supports cache coherency, it transforms memory from a server's private resource into a flexibly schedulable shared resource. This not only allows single machines to overcome physical slot limitations for memory expansion but also enables memory resource pooling across multiple servers, allowing compute nodes to dynamically acquire memory from the shared pool on demand. This breaks through the bottleneck of isolated and limited memory on each server in traditional architectures, significantly improving memory resource utilization efficiency and reducing total cost of ownership.

[0016] Cutting-edge applications in artificial intelligence, such as large language models and deep learning, are constantly pushing boundaries, leading to an exponential expansion in model parameters and a surge in training data. CXL memory pools offer a scalable and cost-effective solution to meet the massive data demands of memory-intensive applications like AI and big data, serving as a cornerstone for building next-generation high-performance, high-resource-utilization data centers.

[0017] The hardware development of CXL memory pools is currently evolving along two clear parallel technical routes: one is the DRAM resource pooling route, which pursues ultimate performance; the other is the heterogeneous media convergence route, which balances capacity and cost. In the DRAM pooling route, the core technology lies in the collaborative design of high-performance CXL switches and controllers, but the cost is too high. In the other heterogeneous media convergence route, besides CXL DRAM devices, CXL hybrid SSDs are the key carrier for realizing the convergence of memory and storage layers. CXL hybrid SSDs typically refer to devices that integrate DRAM cache and NAND flash memory internally, providing byte-addressable space externally through the CXL interface, thus appearing as a byte-addressable memory device. Since the unit capacity cost of DRAM is much higher than that of NAND flash memory (the core storage medium of SSDs), this architecture of hybrid SSDs, with its potential for large-capacity data storage supporting byte addressing, is particularly suitable for AI training, big data analysis, and other applications requiring massive working memory, making it a highly attractive candidate for further reducing the total cost of ownership in new memory systems.

[0018] Currently, the main hardware components of the heterogeneous media convergence CXL memory pool include: CXL switches, which are used to connect hosts to CXL devices or memory pools and are the core for expanding the connection scale; memory carrier devices and controllers, with the controllers mainly responsible for protocol conversion and memory management, while the existing memory carriers mainly include DRAM and CXL hybrid SSDs.

[0019] The synchronous nature of AI applications dictates that the performance bottleneck of communication transactions is highly concentrated at the slowest node in the link. The hierarchical storage architecture and distributed deployment of the CXL memory pool system amplify this latency constraint. For example, compared to the latency of tens to hundreds of nanoseconds for CPU-local DRAM, the latency of CXL-mounted DRAM is 2 to 3 times higher, meaning that even a change in the way the storage medium is mounted can significantly increase latency. Furthermore, the more cost-effective CXL hybrid SSD, even with cache hits, has latency several times that of local DRAM and mounted DRAM. When the DRAM cache misses, the latency can jump by orders of magnitude, surging from hundreds of nanoseconds to microseconds. This uncontrollable latency spike directly restricts the effective aggregation of distributed communication computing power for AI applications.

[0020] Furthermore, in large-scale deployments, CXL memory nodes are not uniformly distributed across each compute unit. They may be located in different parts of the rack or deployed across racks, and the physical distance of signal transmission itself introduces a fixed overhead of tens to hundreds of nanoseconds. More critically, memory access paths may traverse multiple layers of switches, and the cumulative latency caused by multiple switch hops is even more destructive.

[0021] Furthermore, in addition to core factors such as storage medium type, deployment distance, and link hop, dynamic factors such as remaining capacity, node access count, link bandwidth, and contention work together to make latency no longer an attribute of a single device, but a function of the entire network state, all of which can become triggers for latency fluctuations.

[0022] For example, when a CXL node has insufficient remaining capacity, the system needs to initiate data migration or load balancing mechanisms. This process temporarily occupies link resources and increases additional access overhead, leading to increased latency. Uneven access counts can cause hotspot nodes to remain under high load, and even with sufficient link bandwidth, the processing bottlenecks of these hotspot nodes will still slow down overall transaction speed. When a large number of random small read requests consume valuable CXL link bandwidth, or when multiple AI training nodes simultaneously initiate large-scale parameter synchronization or model activation value reading, contention for specific popular memory nodes or critical switching links can instantly trigger traffic congestion, causing a sharp increase in access queuing and latency. Crucially, existing solutions lack a mechanism to perceive the multi-dimensional influencing factors in the CXL memory pool system, often optimizing for a single factor, such as simply increasing link bandwidth or improving hybrid SSD cache prefetch hit rates, while ignoring the coupled effects of capacity, load, and transmission distance. Ultimately, this makes it difficult to effectively control overall system latency and fails to meet the core requirements of low latency and high stability for the strong synchronization of AI services.

[0023] Therefore, the existing CXL hybrid memory pool lacks collaborative perception and management of multi-dimensional factors such as media differences, topology distance, and dynamic load, resulting in high latency and large fluctuations, which has become a performance bottleneck for highly synchronous services such as AI. At the same time, rigid resource scheduling also leads to low utilization efficiency and fails to leverage the cost advantages of the pooling architecture.

[0024] To address the aforementioned technical problems, this application provides a resource scheduling method. To enable those skilled in the art to better understand the solution of this application, the following detailed description, in conjunction with the accompanying drawings and specific embodiments, is provided.

[0025] The specific application environment architecture or specific hardware architecture on which the execution of the resource scheduling method depends is described here.

[0026] Figure 1 This is a schematic diagram of a CXL cluster provided as an embodiment of this application. Figure 1As shown, the CXL memory pool in the CXL cluster includes two types of heterogeneous media memory devices: DRAM and SSD. DRAM represents CXL DRAM devices, and SSD represents CXL hybrid SSD devices. CXL switches are used to build the CXL switching network, enabling physical and logical connections between hosts, CXL memory nodes, and controllers.

[0027] CXL memory nodes are equipped with a dedicated controller responsible for protocol conversion, performance data acquisition, etc., and interact with the host or pool manager through firmware. The host, or compute node, is used to execute computing tasks.

[0028] The pool manager aggregates real-time data from each CXL memory node and reports it to the cluster controller through management interfaces such as the Management Component Transport Protocol (MCTP).

[0029] Figure 2 This is a flowchart illustrating a resource scheduling method provided in an embodiment of this application. This method can be applied to the cluster controller in the CXL memory pool awareness system described above. The cluster controller can be a server or a terminal. The terminal can be a smartphone, PDA, tablet, desktop computer, laptop, all-in-one computer, or other device with data processing capabilities.

[0030] like Figure 2 As shown, the resource scheduling method includes the following steps.

[0031] S201. Obtain the multidimensional attribute data corresponding to multiple candidate nodes.

[0032] Among them, multiple candidate nodes include at least one compute node and multiple memory nodes.

[0033] In some embodiments, the multiple memory nodes are high-speed serial protocol memory nodes in a high-speed serial protocol memory pool.

[0034] A compute node is a node used to perform compute tasks, such as... Figure 1 The host shown contains local DRAM nodes, and memory nodes include CXL memory nodes in the CXL memory pool, such as CXL DRAM devices and CXL hybrid SSD devices.

[0035] Multidimensional attribute data can reflect multiple types of data on the performance, status, and deployment characteristics of candidate nodes, including at least two of the following: candidate node topology location, access path, media type, real-time latency, bandwidth load, and remaining capacity.

[0036] When the host starts up, the firmware traverses the entire CXL switching network, discovers multiple CXL switches and candidate nodes, and assigns them physical address ranges.

[0037] The firmware feeds back the collected information to the cluster controller. Simultaneously, CXL memory nodes report real-time capacity, health status, and performance telemetry data to the pool manager or host via out-of-band or in-band management interfaces, which are then reported by the pool manager to the cluster controller. Thus, the cluster controller obtains multi-dimensional attribute data corresponding to multiple candidate nodes in the memory pool.

[0038] Specifically, the memory pool awareness system combines runtime benchmark tests to calibrate the data reported by the firmware, thereby improving the accuracy of multi-dimensional attribute data.

[0039] S202. Based on the resource requirement information and multi-dimensional attribute data of the target object, determine the target node corresponding to the target object from multiple candidate nodes.

[0040] Wherein, if the access frequency of the target object is greater than the first frequency threshold, the target node corresponding to the target object is determined from the first type of memory nodes among at least one computing node and multiple memory nodes; if the access frequency of the target object is less than or equal to the first frequency threshold, the target node corresponding to the target object is determined from the second type of memory nodes among multiple memory nodes, and the access latency of the first type of memory node is less than the access latency of the second type of memory node.

[0041] The target object refers to the business data or model resources that need to be stored in at least one candidate node, such as model parameters, data pages, etc.

[0042] Resource requirement information refers to the requirements of the target node during the storage and access of the target object, such as the node's capacity, latency, bandwidth, etc.

[0043] The first type of memory node is a memory node with low access latency, such as a high-speed serial protocol memory node integrating dynamic random access memory, i.e., a CXL DRAM device. The second type of memory node is a memory node with higher access latency than the first type, such as a high-speed serial protocol memory node integrating dynamic random access memory and flash memory, i.e., a CXL hybrid SSD device.

[0044] Access frequency refers to the number of times a target object is called or accessed by business logic per unit of time. For example, when the target object includes model parameters of the target model, the access frequency of the target object includes the call frequency of the target model.

[0045] When the access frequency of a target object is greater than the first frequency threshold, it indicates that the target object is more critical and access speed should be prioritized. Therefore, the cluster controller selects the appropriate target node from low-latency compute nodes or first-type memory nodes.

[0046] When the access frequency of the target object is less than or equal to the first frequency threshold, representing the target object's sensitivity to access latency, the cluster controller selects a second type of memory node with high latency but large capacity and low cost, which can release the resources of low-latency nodes for use by target objects with high frequency access, thereby improving the overall utilization of the memory pool.

[0047] Furthermore, the cluster controller selects available target nodes from a specified range of candidate nodes based on the resource requirements information of the target object.

[0048] For example, if resource demand information includes capacity demand information, then it is necessary to remove candidate nodes whose remaining capacity does not meet the capacity demand information from multiple candidate nodes, so as to ensure that the remaining capacity of the target node can meet the capacity demand of the target object.

[0049] S203. Schedule the target object to the target node.

[0050] Specifically, the cluster controller migrates the target object from the meta storage location to the target node.

[0051] This application embodiment allocates frequently accessed target objects to low-latency nodes and infrequently accessed target objects to high-latency nodes, enabling core business or tasks sensitive to access latency to obtain low-latency storage resources, thereby improving the overall resource utilization of the memory pool. While retaining the low-cost advantage of high-latency memory nodes, it reduces the negative impact of access latency and achieves a balance between storage performance and cost.

[0052] In some embodiments, resource requirement information includes the business scenario corresponding to the target object. Based on the resource requirement information of the target object and multi-dimensional attribute data, the target node corresponding to the target object is determined from multiple candidate nodes, including: determining a preset weight for each data item in the multi-dimensional attribute data based on the business scenario; for each memory node among multiple memory nodes, calculating the node performance score corresponding to the memory node according to the preset weight and the multi-dimensional attribute data; and determining at least one memory node whose node performance score is greater than a preset score threshold as the target node corresponding to the target object.

[0053] Business scenarios refer to the application domain or type of task to which the target object belongs, such as AI inference, big data analysis, and distributed training.

[0054] Optionally, the business scenario can be the business scenario marked by the user when submitting the task, or the business scenario automatically determined by the task characteristics.

[0055] Because different business scenarios have different performance preferences for different dimensions of memory nodes, the performance of the same memory node in different business scenarios also has different evaluation standards.

[0056] Preset weights are the assignment of importance values ​​to various indicators in multidimensional attribute data. Different business scenarios have different preset weights, reflecting the degree of preference of the business scenario for different performance indicators.

[0057] The node performance score is calculated by fusing preset weights with multi-dimensional attribute data, which can comprehensively reflect the ability of a memory node to meet the needs of the target object. When the node performance score of a memory node is greater than the preset score threshold, it means that the memory node can meet the resource requirements of the target object in the current business scenario, and therefore it is identified as the target node.

[0058] For example, the multidimensional attribute data includes the average access latency, effective bandwidth, and remaining capacity of the memory node. The node performance score Pi of the memory node is calculated as follows: Pi=α×(1 / Li)+β×Bi+γ×(Ri / Rmax).

[0059] Where Li represents the average access latency of the node, Bi represents the effective bandwidth of the node, Ri represents the remaining capacity of the node, and Rmax represents the maximum node capacity. α, β, and γ are preset weights that are dynamically adjusted according to the application scenario. For example, α is set to 0.4 in AI inference scenarios, and γ is set to 0.3 in big data scenarios.

[0060] This application embodiment transforms abstract business scenarios into computable preset weights, and dynamically determines the weight of node performance scores based on the business scenarios of the scheduled target objects, making node scores closer to business scenarios, improving the rationality and accuracy of node performance score calculation, and further improving the rationality and accuracy of resource scheduling within the cluster.

[0061] In some embodiments, after obtaining the multidimensional attribute data corresponding to the multiple candidate nodes, the method further includes: for each candidate node among the multiple candidate nodes, generating a performance label corresponding to the candidate node based on the multidimensional attribute data.

[0062] Specifically, the multidimensional attribute data includes attribute data of multiple attributes, and the resource requirement information includes requirement information of multiple resources. For each candidate node among multiple candidate nodes, a performance label corresponding to the candidate node is generated based on the multidimensional attribute data, including: determining the target attributes corresponding to multiple resources from the multiple attributes; and generating a performance label corresponding to the candidate node for each candidate node among multiple candidate nodes based on the attribute data of the target attributes.

[0063] Performance tags are identifiers for the performance characteristics of candidate nodes, transforming multi-dimensional attribute data into easily matching tags. For example, for candidate nodes with bandwidth greater than a preset bandwidth threshold, a performance tag of "high bandwidth" is generated.

[0064] The resource requirement information refers to the various performance requirements of the target object for candidate nodes. Target attributes are those attributes selected from the multiple attributes that match the resource requirement information. The system automatically matches the corresponding target attributes by parsing the resource requirement information of the target object, and then filters the attribute data corresponding to the target attributes from the multidimensional attribute data of the candidate nodes to generate corresponding performance labels.

[0065] For example, when the demand information for multiple resources includes the demand for memory capacity but not the demand for bandwidth, only the performance label corresponding to the memory capacity is generated, and the performance label corresponding to the bandwidth is not generated.

[0066] Accordingly, the resource requirement information includes at least one performance requirement label. Based on the resource requirement information of the target object and multi-dimensional attribute data, the target node corresponding to the target object is determined from multiple candidate nodes, including: for each candidate node among the multiple candidate nodes, if the performance label corresponding to the candidate node includes at least one performance requirement label, the candidate node is determined as the target node corresponding to the target object.

[0067] Performance requirement tags are tagged resource requirement information. Optionally, the performance requirement tags should have the same format as the performance tags of candidate nodes for easier matching.

[0068] When the performance tag corresponding to a candidate node includes the performance requirement tag of the target object, that is, when the candidate node has the performance tag required by the target object, it means that the performance of the candidate node can meet the resource requirement information requirements of the target object. Such a candidate node can be used as the target node of the target object.

[0069] This application embodiment generates corresponding performance tags based on the multidimensional attribute data of candidate nodes, and matches them with the performance requirement tags corresponding to the target object. Based on whether the candidate node has the performance tags required by the target object, it can be determined whether the candidate node can be used as the target node, thus improving the efficiency of target node selection.

[0070] In addition, in this embodiment, corresponding performance tags are generated based on the resource requirement information of the target object, avoiding invalid processing of attribute information with low importance and saving the computing resources required to generate tags.

[0071] In some embodiments, the target object includes target model parameters. Before determining the target node corresponding to the target object from multiple candidate nodes based on the resource requirement information and multidimensional attribute data of the target object, the method further includes: determining the target model parameters from multiple candidate model parameters according to the inference request queue corresponding to at least one computing node, wherein the target model parameters are model parameters used to process the next inference request in the inference request queue.

[0072] The inference request queue refers to a queue of inference requests waiting to be processed. At least one compute node's model needs to invoke the corresponding inference resources, processing the inference requests in the queue sequentially. The model itself is a small-scale computational logic and can be directly stored on the local compute node.

[0073] Target model parameters refer to the data used in the inference model to perform inference calculations. They are one of the inference resources, such as weight parameters and bias terms.

[0074] Based on the information of the next inference request in the inference request queue, the target model parameter is determined from multiple candidate model parameters, which may be stored on different memory nodes.

[0075] Accordingly, the target object is scheduled to the target node, including: scheduling the target model parameters to the target node before starting to process the next inference request.

[0076] In other words, the target model parameters have been migrated to the target node before the next inference request is processed. Therefore, when processing the next inference request, the compute node can directly call the target model parameters in the target node.

[0077] This application embodiment determines the target model parameters required to process the next inference request in advance, and migrates the target model parameters to the target node before starting to process the next inference request. This allows the computing node to directly call the target model parameters from the target node with matching performance when starting to process the next inference request, without waiting for the parameters to load, thereby improving the processing efficiency of the inference task and further enhancing the utilization of memory resources.

[0078] In some embodiments, after obtaining the multidimensional attribute data corresponding to the multiple candidate nodes respectively, the method further includes: transmitting the multidimensional attribute data corresponding to the multiple candidate nodes to the target application based on a preset interface, wherein the target application is used to update the parameters of the memory allocator based on the multidimensional attribute data.

[0079] The target application refers to the business application that requires memory resources, and the parameters of the memory allocator refer to the parameters used inside the application to decide the memory allocation strategy, such as the priority of node selection, the storage rules of target objects, and the threshold judgment criteria for various data.

[0080] The updated parameters of the memory allocator will affect subsequent resource scheduling processes. For example, the target application can bind certain high-performance target objects to a specific target node based on multi-dimensional attribute data that meets certain requirements, or update the corresponding resource requirement information for certain target objects. For instance, for databases and AI inference applications that are aware of data access patterns, it can be explicitly declared that indexes, model parameters, etc., are bound to compute nodes. Once bound, the indexes and model parameters will not participate in cluster resource scheduling for a preset period of time.

[0081] This application embodiment transmits the multi-dimensional attribute data corresponding to multiple candidate nodes to the target application, enabling the application to update its memory allocation strategy based on the current memory resources. This achieves memory resource planning and efficient utilization at the application level, and, combined with node-level resource scheduling, improves the synergy of resource scheduling.

[0082] In some embodiments, determining the target node corresponding to the target object from multiple candidate nodes based on the resource requirement information and multidimensional attribute data of the target object includes: generating an analysis task based on the resource requirement information and multidimensional attribute data of the target object; sending the analysis task to a preset task processing hardware; and receiving the analysis results sent by the task processing hardware, wherein the analysis results include the node identifier of the target node.

[0083] Preset task processing hardware refers to dedicated hardware devices corresponding to multiple memory nodes, such as network cards.

[0084] The analysis task carries resource requirement information and multi-dimensional attribute data, which are used to instruct the preset task processing hardware to select a target node that meets the requirements of the resource requirement information from multiple candidate nodes based on the resource requirement information and multi-dimensional attribute data. This is equivalent to offloading the analysis and calculation process of selecting the target node to the preset task processing hardware, which provides the node identifier of the target node so that the system can determine the target node based on the node identifier and schedule the target object to the target node.

[0085] This application embodiment offloads the analysis and calculation process of selecting target nodes to preset task processing hardware, and reduces the time spent selecting target nodes by utilizing the dedicated computing power and parallel processing capabilities of the hardware device, thereby quickly determining the target node and further improving the efficiency of resource scheduling.

[0086] In some embodiments, the method further includes: migrating the target object to an idle node among a plurality of candidate nodes in the event of a failure of the target node.

[0087] Optionally, if the number of failed access attempts to the target object in the target node exceeds a preset threshold, the target node is determined to be faulty.

[0088] Access failures include, but are not limited to, connection timeouts, data transmission errors, and non-response from the target node. When an access fails, the cumulative number of failed accesses to the target node is recorded.

[0089] When the number of failed accesses exceeds a preset threshold, it indicates that the target node has failed and cannot continue to handle access operations. The target node is marked as faulty and / or unavailable, and the target objects stored on it are migrated to other idle nodes.

[0090] Optionally, the target node can be re-determined based on the resource scheduling method of the above embodiments, and the target object can be migrated. Alternatively, the target object can be migrated to a pre-deployed backup node and await rescheduling.

[0091] Optionally, a preset threshold number of times can be determined based on the current business scenario.

[0092] This application embodiment ensures normal access to target objects by promptly migrating the target objects stored on a target node to an idle node when the target node fails, thus avoiding business interruption or data loss due to node failure and guaranteeing the reliability of the resource scheduling system.

[0093] In some embodiments, after obtaining the multidimensional attribute data corresponding to the multiple candidate nodes, the method further includes: creating a non-uniform memory access topology graph corresponding to the multiple candidate nodes, wherein the topology nodes in the non-uniform memory access topology graph correspond one-to-one with the candidate nodes, and the node information of the topology nodes includes multidimensional attribute data.

[0094] Non-uniform memory access topology is a visual graphical model based on NUMA architecture that describes the physical connections and logical relationships between candidate nodes. It is used to intuitively present the distribution and association status of candidate nodes, and at the same time present the structured data of the multi-dimensional attribute data of each candidate node.

[0095] Topology nodes are virtual vertices in the topology graph, corresponding to physically existing candidate nodes. They include not only multiple memory nodes but also computing nodes. Thus, the topological relationships and multi-dimensional attribute data between selectable nodes are integrated into a unified NUMA topology graph. When performing resource scheduling, relevant information about global candidate nodes can be quickly obtained, reducing information query overhead and improving resource scheduling efficiency.

[0096] Optionally, the latency data of candidate nodes can be converted into relative distance values ​​to construct a NUMA distance matrix. The distance value corresponding to a candidate node is positively correlated with its access latency; the lower the latency, the smaller the distance value. Simultaneously, the connection edges in the NUMA topology graph are defined to represent the physical or communication connection relationships between candidate nodes. The access distance is quantified by the weights of the connection edges, which are the sum of the base latency, the penalty value corresponding to the distance value, and the link hop penalty value. The penalty value corresponding to the distance value is positively correlated with the distance value, and the link hop penalty value is positively correlated with the number of hops (e.g., the number of times a switch is traversed) required to access the candidate node.

[0097] Therefore, the access cost of any candidate node can be determined based on the weight of the connecting edges. The higher the weight, the higher the access cost. Furthermore, this cost includes not only the base latency but also the impact of the number of link hops, providing a more comprehensive reflection of the actual access cost and offering a more complete data reference for resource scheduling. Simultaneously, by integrating the access cost into the NUMA topology graph, information query overhead can be further reduced, improving resource scheduling efficiency.

[0098] The following section introduces the scheduling methods for memory resources in the context of actual business scenarios. Figure 3 This is a schematic diagram of the structure of a CXL memory pool awareness system provided in an embodiment of this application. Figure 3 As shown, the system is divided into a cluster control layer, a system layer, and a hardware or firmware layer.

[0099] The hardware or firmware layer consists of CXL switches, CXL memory devices and their controllers, servers (hosts), and firmware. The firmware is used to sense data from candidate nodes and build corresponding Advanced Configuration and Power Interface (ACPI) tables. The data sensed by the firmware includes, but is not limited to, the capacity, location, performance, latency, topology, access path, media type, bandwidth, dynamic information, and status information of the memory nodes.

[0100] The system layer includes the system kernel, which contains an ACPI parser, a page migration module, a Non-Uniform Memory Access (NUMA) subsystem, and a memory management and scheduler. It parses the ACPI table provided by the firmware, transforming information perceived by the hardware or firmware layer into abstract data usable by the kernel and applications, constructing a NUMA topology graph, and executing resource management and scheduling tasks for a single host and its corresponding CXL memory nodes based on this graph. The cluster control layer includes a global view cluster management controller, cluster plugins, and a scheduler. It integrates data from the system layer to obtain global topology, performance, and load data, and performs scheduling optimization based on this data, controlling the system layer to perform resource management and scheduling for multiple hosts and global CXL memory nodes.

[0101] The data for the above candidate nodes can be divided into the following dimensions: The capacity and location awareness dimension includes information such as the candidate nodes in the memory pool, the node capacity, and location information. The firmware reports each CXL memory node as an independent NUMA node, allowing the system to enumerate these nodes and know their physical address range upon startup. The firmware manages this information through the ACPI table, a data structure created by the computer firmware and placed in a specific location in memory. Its core function is to clearly describe the layout of the entire hardware platform, hardware performance attributes and capabilities, and power management rules to the operating system.

[0102] Performance and latency awareness dimensions, including the speed of accessing different candidate nodes, can be quantified using a NUMA distance matrix, or dynamically calibrated through runtime benchmarking, enabling the system to have the ability to perceive and correct information in real time.

[0103] Topology and path awareness dimensions include information about the physical and logical paths from the local node to any candidate node. First, during startup, the firmware and host system can utilize the CXL protocol's discovery and enumeration mechanisms to construct a system topology graph; a global view can then be obtained from a higher-level centralized management entity, such as a cluster management controller.

[0104] The media and bandwidth awareness dimension includes memory type and bandwidth information for different candidate nodes. Detailed performance information, including read and write latency, bandwidth, and memory type such as DRAM or NAND, is recorded through entries in the ACPI table.

[0105] The dynamic and state-aware dimension includes the memory allocation status, health status, and real-time load in the memory pool. Real-time telemetry data, such as remaining capacity, access count, and error rate, can be obtained from the pool manager through CXL management interfaces such as MCTP and vendor-defined command interfaces.

[0106] Among them, MCTP is a lightweight, message-oriented communication protocol that defines how the management controller communicates in a standardized manner with various hardware components, including CXL devices, SSDs, network cards, etc.

[0107] In some embodiments, an online video content moderation platform needs to run multiple AI models simultaneously to perform functions such as violation detection, violent scene recognition, and text recognition. These models vary in size, are invoked at different frequencies, and are sensitive to latency in the inference process.

[0108] A heterogeneous memory pool consisting of CXL DRAM devices and CXL hybrid SSD devices is constructed. Each CXL node controller integrates a hardware performance counting unit and reports its own media type, remaining capacity, and current read latency to the memory pool awareness system in real time through the MCTP interface.

[0109] The system kernel driver identifies each memory node as an independent node and continuously monitors the access frequency of the graphics processing unit (GPU) of the compute node to the model memory. If a model memory page has a low access frequency, for example, if the number of accesses within 2 seconds is less than a threshold, it is marked as a cold page and transparently migrated to the CXL hybrid SSD device via the CXL protocol. If the access frequency of a model memory page surges, it is prefetched to the compute node or migrated to the CXL DRAM device.

[0110] The cluster management controller abstracts CXL node resources into two categories: fast (DRAM) and slow (CXL-SSD). The scheduler uses fast CXL nodes as the working set for calculating the weights of the currently hot model, and slow CXL nodes as the persistent storage pool for all model parameters. Based on the real-time remaining capacity and node latency labels of fast CXL nodes, the scheduler dynamically schedules the most active model parameters from slow CXL nodes to fast CXL nodes with the fastest access path.

[0111] During model loading, the kernel is informed that one or more CXL nodes will be used to store the model parameters of the AI ​​model, and based on the inference request queue, it predictively prompts the kernel to migrate the next required model parameters to the fast class CXL node in advance.

[0112] When a CXL DRAM node is overloaded, the cluster management controller will reschedule some model parameters, especially those with access frequencies higher than the preset frequency, to idle nodes to achieve cluster-level load balancing.

[0113] In another embodiment, a distributed training task needs to be deployed, which includes a parameter server and multiple training workers.

[0114] The parameter server is responsible for maintaining and updating a huge number of model parameters, and its memory requirements are greater than its access latency.

[0115] Multiple training workers are responsible for calculating gradients and need to frequently read parameters from the parameter server. Therefore, they are very sensitive to the access latency of the parameter server's memory. The synchronization speed of the entire training task depends on the speed at which the slowest training worker reads parameters.

[0116] The cluster consists of four compute nodes, designated Node-A, Node-B, Node-C, and Node-D. They share a remote CXL memory pool via a two-layer CXL switching network. The CXL memory pool provides low-latency CXL DRAM devices and high-capacity CXL hybrid SSD devices as memory nodes.

[0117] The cluster management controller uses multi-dimensional attribute parameters of each node, combined with dynamic benchmark tests, to plot access latency graphs and generate performance labels for each compute node. Node-A has the lowest latency to CXL DRAM devices, assumed to be 150ns; Node-B has medium latency, assumed to be 200ns; Node-C has higher latency, but the path to high-capacity CXL hybrid SSD devices is optimal; and Node-D has medium latency, assumed to be 200ns.

[0118] The parameter server's resource requirements indicate that it needs a large-capacity memory node that can tolerate high latency, for example, 800G. The training worker's resource requirements indicate that it requests a low-latency memory node and uses the node selection function to force scheduling to a low-latency node.

[0119] Therefore, for the parameter server, it is necessary to find a memory node with a remaining capacity of more than 800G. Assuming that Node-C meets the requirements, although its latency is higher, the parameter server is not sensitive to access latency, so the parameter server is scheduled to Node-C.

[0120] For the training worker, it needs to be scheduled to a memory node with a low-latency label and sufficient remaining capacity. Assuming that all nodes meet the remaining capacity requirements of the training worker, but only Node-A has a low-latency label, the training worker will be scheduled to Node-A.

[0121] This application's embodiments construct a multi-dimensional perception system through full-stack collaboration of hardware, firmware, system, and cluster control, overcoming the latency bottleneck of pooled architecture and optimizing memory resource utilization. Based on the multi-dimensional attribute information of candidate nodes, including but not limited to capacity location, performance latency, and topology path, it addresses various factors affecting access latency, such as storage media type, deployment distance, and load status. Combined with dynamic strategies for migrating high-frequency access objects to low-latency nodes and low-frequency access objects to high-capacity nodes, it adapts to the highly synchronous business needs of AI inference and distributed training. Simultaneously, it achieves cross-node memory sharing and hierarchical management of heterogeneous media, improving the utilization of idle resources, reducing total cost of ownership while ensuring core performance, and avoiding hotspot node degradation through cluster-level load balancing.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0123] Embodiments of this application also provide a resource scheduling device, including an acquisition module, a first determination module, and a storage module; wherein, the acquisition module is used to acquire multi-dimensional attribute data corresponding to multiple candidate nodes, the multiple candidate nodes including at least one computing node and multiple memory nodes; the first determination module is used to determine the target node corresponding to the target object from the multiple candidate nodes based on the resource requirement information of the target object and the multi-dimensional attribute data; wherein, when the access frequency of the target object is greater than a first frequency threshold, the target node corresponding to the target object is determined from a first type of memory node among the at least one computing node and the multiple memory nodes; when the access frequency of the target object is less than or equal to the first frequency threshold, the target node corresponding to the target object is determined from a second type of memory node among the multiple memory nodes, the access latency of the first type of memory node is less than the access latency of the second type of memory node; the storage module is used to schedule the target object to the target node.

[0124] In some embodiments, the resource requirement information includes the business scenario corresponding to the target object. The first determining module includes a first determining unit, a calculation unit, and a second determining unit. The first determining unit is used to determine the preset weight of each data item in the multidimensional attribute data based on the business scenario. The calculation unit is used to calculate the node performance score corresponding to each memory node in the multiple memory nodes according to the preset weight and the multidimensional attribute data. The second determining unit is used to determine at least one memory node whose node performance score is greater than the preset score threshold as the target node corresponding to the target object.

[0125] In some embodiments, the target object includes the model parameters of the target model, and the access frequency of the target object includes the call frequency of the target model.

[0126] In some embodiments, the resource scheduling device further includes a tag generation module, which generates a performance tag corresponding to each candidate node based on multidimensional attribute data for each candidate node among a plurality of candidate nodes.

[0127] In some embodiments, the multidimensional attribute data includes attribute data of multiple attributes, the resource requirement information includes requirement information of multiple resources, and the tag generation module is further used to determine the target attribute corresponding to the multiple resources from the multiple attributes; for each candidate node among multiple candidate nodes, a performance tag corresponding to the candidate node is generated based on the attribute data of the target attribute.

[0128] In some embodiments, the resource requirement information includes at least one performance requirement label, and the first determining module is further configured to determine, for each of the multiple candidate nodes, if the performance label corresponding to the candidate node includes at least one performance requirement label, the candidate node is the target node corresponding to the target object.

[0129] In some embodiments, the target object includes target model parameters, and the resource scheduling device further includes a second determining module, configured to determine the target model parameters from multiple candidate model parameters based on the inference request queue corresponding to at least one computing node, wherein the target model parameters are model parameters used to process the next inference request in the inference request queue.

[0130] In some embodiments, the storage module is also configured to schedule the target model parameters to the target node before starting to process the next inference request.

[0131] In some embodiments, the resource requirement information includes capacity requirement information, and the first determining module is further configured to remove candidate nodes whose remaining capacity does not meet the capacity requirement information from among multiple candidate nodes.

[0132] In some embodiments, the resource scheduling device further includes a transmission module, which is used to transmit multi-dimensional attribute data corresponding to multiple candidate nodes to a target application based on a preset interface, and the target application is used to update the parameters of the memory allocator based on the multi-dimensional attribute data.

[0133] In some embodiments, the first determining module further includes an unloading unit, configured to generate an analysis task based on the resource requirement information and multidimensional attribute data of the target object; send the analysis task to a preset task processing hardware; and receive the analysis results sent by the task processing hardware, the analysis results including the node identifier of the target node.

[0134] In some embodiments, the resource scheduling apparatus further includes a migration module for migrating the target object to an idle node among multiple candidate nodes in the event of a target node failure.

[0135] In some embodiments, the migration module includes a fault diagnosis unit, used to determine that the target node has failed if the number of failed accesses to the target object in the target node exceeds a preset threshold.

[0136] In some embodiments, the resource scheduling device further includes a creation module for creating a non-uniform memory access topology graph corresponding to multiple candidate nodes. The topology nodes in the non-uniform memory access topology graph correspond one-to-one with the candidate nodes, and the node information of the topology nodes includes multi-dimensional attribute data.

[0137] In some embodiments, the multiple memory nodes are high-speed serial protocol memory nodes in a high-speed serial protocol memory pool.

[0138] In some embodiments, the first type of memory node includes a high-speed serial protocol memory node with integrated dynamic random access memory, and the second type of memory node includes a high-speed serial protocol memory node with integrated dynamic random access memory and flash memory.

[0139] In some embodiments, the multidimensional attribute data includes at least two of the following: the topological location of the candidate node, the access path, the media type, the real-time latency, the bandwidth load, and the remaining capacity.

[0140] For a description of the features in the embodiment corresponding to the resource scheduling device, please refer to the relevant description in the embodiment corresponding to the resource scheduling method, which will not be repeated here.

[0141] This application embodiment allocates frequently accessed target objects to low-latency nodes and infrequently accessed target objects to high-latency nodes, enabling core business or tasks sensitive to access latency to obtain low-latency storage resources, thereby improving the overall resource utilization of the memory pool. While retaining the low-cost advantage of high-latency memory nodes, it reduces the negative impact of access latency and achieves a balance between storage performance and cost.

[0142] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the resource scheduling method embodiments described above.

[0143] This application embodiment allocates frequently accessed target objects to low-latency nodes and infrequently accessed target objects to high-latency nodes, enabling core business or tasks sensitive to access latency to obtain low-latency storage resources, thereby improving the overall resource utilization of the memory pool. While retaining the low-cost advantage of high-latency memory nodes, it reduces the negative impact of access latency and achieves a balance between storage performance and cost.

[0144] Meanwhile, by transforming abstract business scenarios into computable preset weights, and dynamically determining the weight of node performance scores based on the business scenarios of the scheduled target objects, node scores are made closer to business scenarios, improving the rationality and accuracy of node performance score calculation, and further enhancing the rationality and accuracy of resource scheduling within the cluster.

[0145] Furthermore, by generating corresponding performance labels based on the multidimensional attribute data of candidate nodes and matching them with the performance requirement labels corresponding to the target object, it is possible to determine whether a candidate node can be used as a target node based on whether it possesses the performance labels required by the target object, thus improving the efficiency of target node selection. Targeted generation of corresponding performance labels based on the resource requirement information of the target object avoids invalid processing of less important attribute information and saves the computational resources required for label generation.

[0146] Furthermore, by pre-determining the target model parameters required to process the next inference request and migrating them to the target node before starting to process the next inference request, the compute node can directly call the target model parameters from the performance-matched target node when starting to process the next inference request, without waiting for the parameters to load, thereby improving the processing efficiency of the inference task and further enhancing the utilization of memory resources.

[0147] In some embodiments, the processor is configured to run computer programs to perform: The process involves acquiring multi-dimensional attribute data corresponding to multiple candidate nodes, including at least one compute node and multiple memory nodes; determining the target node corresponding to the target object from the multiple candidate nodes based on the resource requirement information and multi-dimensional attribute data of the target object; wherein, if the access frequency of the target object is greater than a first frequency threshold, the target node corresponding to the target object is determined from the first type of memory nodes among the at least one compute node and the multiple memory nodes; if the access frequency of the target object is less than or equal to the first frequency threshold, the target node corresponding to the target object is determined from the second type of memory nodes among the multiple memory nodes, wherein the access latency of the first type of memory nodes is less than the access latency of the second type of memory nodes; and scheduling the target object to the target node.

[0148] The processor is configured to run a computer program to perform: determining a preset weight for each data item in the multidimensional attribute data based on the business scenario; calculating the node performance score corresponding to each memory node in the multiple memory nodes according to the preset weight and the multidimensional attribute data; and determining at least one memory node whose node performance score is greater than the preset score threshold as the target node corresponding to the target object.

[0149] The target object includes the model parameters of the target model, and the access frequency of the target object includes the call frequency of the target model.

[0150] The processor is configured to run a computer program to perform the following: for each of a plurality of candidate nodes, generate a performance label corresponding to the candidate node based on multidimensional attribute data.

[0151] The processor is configured to run a computer program to perform: determining target attributes corresponding to multiple resources from multiple attributes; and for each candidate node among multiple candidate nodes, generating a performance label corresponding to the candidate node based on the attribute data of the target attributes.

[0152] The processor is configured to run a computer program to perform the following: for each of a plurality of candidate nodes, if the performance label corresponding to the candidate node includes at least one performance requirement label, determine the candidate node as the target node corresponding to the target object.

[0153] The processor is configured to run a computer program to perform: determining a target model parameter from multiple candidate model parameters based on an inference request queue corresponding to at least one computing node, the target model parameter being the model parameter used to process the next inference request in the inference request queue.

[0154] The processor is configured to run computer programs to perform: scheduling target model parameters to the target node before starting to process the next inference request.

[0155] The processor is configured to run a computer program to remove candidate nodes whose remaining capacity does not meet the capacity requirements from among multiple candidate nodes.

[0156] The processor is configured to run a computer program to execute: passing multidimensional attribute data corresponding to multiple candidate nodes to a target application based on a preset interface, and the target application uses the multidimensional attribute data to update the parameters of the memory allocator.

[0157] The processor is configured to run computer programs to perform: generating analysis tasks based on the resource requirements information and multidimensional attribute data of the target object; sending the analysis tasks to preset task processing hardware; and receiving the analysis results sent by the task processing hardware, including the node identifier of the target node.

[0158] The processor is configured to run a computer program to perform the following: in the event of a failure of the target node, migrate the target object to an idle node among multiple candidate nodes.

[0159] The processor is configured to run a computer program to determine if the target node is faulty if the number of failed access attempts to the target object in the target node exceeds a preset threshold.

[0160] The processor is configured to run a computer program to perform: creating a non-uniform memory access topology graph corresponding to multiple candidate nodes, wherein the topology nodes in the non-uniform memory access topology graph correspond one-to-one with the candidate nodes, and the node information of the topology nodes includes multi-dimensional attribute data.

[0161] Multiple memory nodes are high-speed serial protocol memory nodes within a high-speed serial protocol memory pool. Type I memory nodes include high-speed serial protocol memory nodes with integrated dynamic random access memory (DRAM), while Type II memory nodes include high-speed serial protocol memory nodes with integrated DRAM and flash memory. Multidimensional attribute data includes at least two of the following: candidate node topology location, access path, media type, real-time latency, bandwidth load, and remaining capacity.

[0162] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described resource scheduling method embodiments at runtime.

[0163] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0164] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described resource scheduling method embodiments.

[0165] This application embodiment allocates frequently accessed target objects to low-latency nodes and infrequently accessed target objects to high-latency nodes, enabling core business or tasks sensitive to access latency to obtain low-latency storage resources, thereby improving the overall resource utilization of the memory pool. While retaining the low-cost advantage of high-latency memory nodes, it reduces the negative impact of access latency and achieves a balance between storage performance and cost.

[0166] Meanwhile, by transforming abstract business scenarios into computable preset weights, and dynamically determining the weight of node performance scores based on the business scenarios of the scheduled target objects, node scores are made closer to business scenarios, improving the rationality and accuracy of node performance score calculation, and further enhancing the rationality and accuracy of resource scheduling within the cluster.

[0167] Furthermore, by generating corresponding performance labels based on the multidimensional attribute data of candidate nodes and matching them with the performance requirement labels corresponding to the target object, it is possible to determine whether a candidate node can be used as a target node based on whether it possesses the performance labels required by the target object, thus improving the efficiency of target node selection. Targeted generation of corresponding performance labels based on the resource requirement information of the target object avoids invalid processing of less important attribute information and saves the computational resources required for label generation.

[0168] Furthermore, by pre-determining the target model parameters required to process the next inference request and migrating them to the target node before starting to process the next inference request, the compute node can directly call the target model parameters from the performance-matched target node when starting to process the next inference request, without waiting for the parameters to load, thereby improving the processing efficiency of the inference task and further enhancing the utilization of memory resources.

[0169] When the computer program is executed by the processor, it performs the following: acquiring multi-dimensional attribute data corresponding to multiple candidate nodes, including at least one computing node and multiple memory nodes; determining the target node corresponding to the target object from the multiple candidate nodes based on the resource requirement information and multi-dimensional attribute data of the target object; wherein, if the access frequency of the target object is greater than a first frequency threshold, determining the target node corresponding to the target object from a first type of memory node among the at least one computing node and the multiple memory nodes; if the access frequency of the target object is less than or equal to the first frequency threshold, determining the target node corresponding to the target object from a second type of memory node among the multiple memory nodes, wherein the access latency of the first type of memory node is less than the access latency of the second type of memory node; and scheduling the target object to the target node.

[0170] When the computer program is executed by the processor, it implements the following: determining the preset weight of each data item in the multidimensional attribute data based on the business scenario; for each memory node among multiple memory nodes, calculating the node performance score corresponding to the memory node according to the preset weight and the multidimensional attribute data; and determining at least one memory node whose node performance score is greater than the preset score threshold as the target node corresponding to the target object.

[0171] The target object includes the model parameters of the target model, and the access frequency of the target object includes the call frequency of the target model.

[0172] When a computer program is executed by a processor, it achieves the following: for each of multiple candidate nodes, a performance label corresponding to the candidate node is generated based on multidimensional attribute data.

[0173] When a computer program is executed by a processor, it performs the following: determining the target attributes corresponding to multiple resources from multiple attributes; and for each candidate node among multiple candidate nodes, generating a performance label corresponding to the candidate node based on the attribute data of the target attributes.

[0174] When a computer program is executed by a processor, it implements the following: For each of a plurality of candidate nodes, if the performance label corresponding to the candidate node includes at least one performance requirement label, the candidate node is determined to be the target node corresponding to the target object.

[0175] When the computer program is executed by the processor, it performs the following: Based on the inference request queue corresponding to at least one computing node, it determines the target model parameter from multiple candidate model parameters. The target model parameter is the model parameter used to process the next inference request in the inference request queue.

[0176] When a computer program is executed by a processor, it schedules the target model parameters to the target node before starting to process the next inference request.

[0177] When a computer program is executed by a processor, it removes candidate nodes from a pool of candidate nodes whose remaining capacity does not meet the capacity requirements.

[0178] When the computer program is executed by the processor, it implements the following: Based on a preset interface, it transmits the multi-dimensional attribute data corresponding to multiple candidate nodes to the target application, which then uses the multi-dimensional attribute data to update the parameters of the memory allocator.

[0179] When the computer program is executed by the processor, it performs the following: generating an analysis task based on the resource requirement information and multi-dimensional attribute data of the target object; sending the analysis task to the preset task processing hardware; and receiving the analysis results sent by the task processing hardware, including the node identifier of the target node.

[0180] When a computer program is executed by a processor, it achieves the following: in the event of a failure in the target node, the target object is migrated to an idle node among multiple candidate nodes.

[0181] When a computer program is executed by a processor, it determines that the target node has failed if the number of failed access attempts to the target object in the target node exceeds a preset threshold.

[0182] When a computer program is executed by a processor, it achieves the following: creating a non-uniform memory access topology graph corresponding to multiple candidate nodes. The topology nodes in the non-uniform memory access topology graph correspond one-to-one with the candidate nodes, and the node information of the topology nodes includes multi-dimensional attribute data.

[0183] Multiple memory nodes are high-speed serial protocol memory nodes within a high-speed serial protocol memory pool. Type I memory nodes include high-speed serial protocol memory nodes with integrated dynamic random access memory (DRAM), while Type II memory nodes include high-speed serial protocol memory nodes with integrated DRAM and flash memory. Multidimensional attribute data includes at least two of the following: candidate node topology location, access path, media type, real-time latency, bandwidth load, and remaining capacity.

[0184] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described resource scheduling method embodiments.

[0185] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0186] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0187] The resource scheduling method, apparatus, readable storage medium, and computer program product provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A resource scheduling method, characterized in that, The method includes: Obtain multidimensional attribute data corresponding to multiple candidate nodes, wherein the multiple candidate nodes include at least one computing node and multiple memory nodes; Based on the resource requirement information of the target object and the multidimensional attribute data, the target node corresponding to the target object is determined from the plurality of candidate nodes; wherein, if the access frequency of the target object is greater than a first frequency threshold, the target node corresponding to the target object is determined from the first type of memory node among the at least one computing node and the plurality of memory nodes; if the access frequency of the target object is less than or equal to the first frequency threshold, the target node corresponding to the target object is determined from the second type of memory node among the plurality of memory nodes, wherein the access latency of the first type of memory node is less than the access latency of the second type of memory node; The target object is scheduled to the target node.

2. The resource scheduling method according to claim 1, characterized in that, The resource requirement information includes the business scenario corresponding to the target object. The process of determining the target node corresponding to the target object from the plurality of candidate nodes based on the resource requirement information of the target object and the multi-dimensional attribute data includes: Based on the business scenario, determine the preset weight of each data item in the multidimensional attribute data; For each of the plurality of memory nodes, a node performance score is calculated based on the preset weight and the multidimensional attribute data. At least one memory node whose node performance score is greater than a preset score threshold is identified as the target node corresponding to the target object.

3. The resource scheduling method according to claim 1, characterized in that, The target object includes the model parameters of the target model, and the access frequency of the target object includes the call frequency of the target model.

4. The resource scheduling method according to claim 1, characterized in that, After obtaining the multidimensional attribute data corresponding to the multiple candidate nodes, the method further includes: For each of the plurality of candidate nodes, a performance label corresponding to the candidate node is generated based on the multidimensional attribute data.

5. The resource scheduling method according to claim 4, characterized in that, The multidimensional attribute data includes attribute data of multiple attributes, the resource requirement information includes requirement information of multiple resources, and for each candidate node among the multiple candidate nodes, a performance label corresponding to the candidate node is generated based on the multidimensional attribute data, including: Determine the target attribute corresponding to the multiple resources from the multiple attributes; For each of the plurality of candidate nodes, a performance label corresponding to the candidate node is generated based on the attribute data of the target attribute.

6. The resource scheduling method according to claim 4, characterized in that, The resource requirement information includes at least one performance requirement tag. The process of determining the target node corresponding to the target object from the plurality of candidate nodes based on the resource requirement information of the target object and the multi-dimensional attribute data includes: For each of the plurality of candidate nodes, if the performance label corresponding to the candidate node includes at least one performance requirement label, the candidate node is determined to be the target node corresponding to the target object.

7. The resource scheduling method according to claim 1, characterized in that, The target object includes target model parameters. Before determining the target node corresponding to the target object from the plurality of candidate nodes based on the resource requirement information of the target object and the multi-dimensional attribute data, the method further includes: Based on the inference request queue corresponding to the at least one computing node, the target model parameter is determined from multiple candidate model parameters, and the target model parameter is the model parameter used to process the next inference request in the inference request queue.

8. The resource scheduling method according to claim 7, characterized in that, The step of scheduling the target object to the target node includes: Before starting to process the next inference request, the target model parameters are scheduled to the target node.

9. The resource scheduling method according to claim 1, characterized in that, The resource demand information includes capacity demand information. The process of determining the target node corresponding to the target object from the plurality of candidate nodes based on the resource demand information of the target object and the multi-dimensional attribute data includes: Remove candidate nodes from the plurality of candidate nodes whose remaining capacity does not meet the capacity requirement information.

10. The resource scheduling method according to claim 1, characterized in that, The method further includes: Based on a preset interface, the multidimensional attribute data corresponding to the multiple candidate nodes are transmitted to the target application, and the target application is used to update the parameters of the memory allocator based on the multidimensional attribute data.

11. The resource scheduling method according to claim 1, characterized in that, The process of determining the target node corresponding to the target object from the plurality of candidate nodes based on the resource requirement information of the target object and the multi-dimensional attribute data includes: An analysis task is generated based on the resource requirement information of the target object and the multidimensional attribute data. The analysis task is sent to the preset task processing hardware; The system receives analysis results sent by the task processing hardware, the analysis results including the node identifier of the target node.

12. The resource scheduling method according to claim 1, characterized in that, The method further includes: In the event of a failure in the target node, the target object will be migrated to an idle node among the multiple candidate nodes.

13. The resource scheduling method according to claim 12, characterized in that, Before migrating the target object to an idle node among the plurality of candidate nodes in the event of a failure of the target node, the method further includes: If the number of failed access attempts to the target object in the target node exceeds a preset threshold, the target node is determined to be faulty.

14. The resource scheduling method according to claim 1, characterized in that, After obtaining the multidimensional attribute data corresponding to multiple candidate nodes, the method further includes: A non-uniform memory access topology graph corresponding to the multiple candidate nodes is created. The topology nodes in the non-uniform memory access topology graph correspond one-to-one with the candidate nodes. The node information of the topology nodes includes the multi-dimensional attribute data.

15. The resource scheduling method according to claim 1, characterized in that, The multiple memory nodes are high-speed serial protocol memory nodes in the high-speed serial protocol memory pool.

16. The resource scheduling method according to claim 15, characterized in that, The first type of memory node includes a high-speed serial protocol memory node with integrated dynamic random access memory, and the second type of memory node includes a high-speed serial protocol memory node with integrated dynamic random access memory and flash memory.

17. The resource scheduling method according to claim 1, characterized in that, The multidimensional attribute data includes at least two of the following: The candidate node's topological location, access path, media type, real-time latency, bandwidth load, and remaining capacity.

18. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the resource scheduling method as described in any one of claims 1 to 17 when executing the computer program.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the resource scheduling method as described in any one of claims 1 to 17.

20. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the resource scheduling method as described in any one of claims 1 to 17.

Images